xAIVerified interview challenge
Improve classifier precision with a small team
Build a credible plan to improve classifier precision without losing recall or contaminating the evaluation.
Unlock with Full AccessThe challenge
A small team has three weeks to improve the precision of a classifier that flags potentially miscalibrated scientific claims, without allowing recall to fall below its required floor. Contractor labelers are available, but review capacity is limited.
Plan the measurement audit, labeling work and iteration needed to make a credible recommendation to the CEO. The available levers concern data, taxonomy, sampling and thresholds, so the challenge is to organize a defensible improvement programme around the existing model.
What you'll practise
Labelling operations
Set up calibration, arbitration and accountability for a mixed-experience contractor team.
Measurement integrity
Keep evaluation evidence interpretable as labels, thresholds and training data change.
Delivery commitments
Connect the plan to measurable progress and conditions for changing the target.
Write your reasoning and submit it for feedback.