AnthropicVerified interview challenge
Training-data pipeline bug hunt
Find data-preparation bugs that can produce convincing but unreliable model evaluation results.
Unlock with Full AccessThe challenge
A pipeline splits records, fits a vectorizer, shuffles examples and evaluates a held-out set. The contract requires accounts to stay in one split and features to remain paired with their labels. Review the implementation for leakage, corrupted examples and incomplete validation, then explain how to correct them.
What you'll practise
Data separation
Check whether preprocessing and splitting preserve the boundary around held-out data.
Example integrity
Trace record identity and feature-label pairs through the pipeline.
Evaluation coverage
Verify that the reported result accounts for the complete validation set.
Data preparation pipelineCode included in the full challenge
Your pipeline review
Connect each code fault to its effect on training or evaluation.
The code, instructions and feedback are included with Full Access.