Anthropic
OpenAI
PerplexityBased on real interviews
Model-adaptation decision: document-extraction API
Decide how to improve a document-extraction API when accuracy varies sharply between customers.
Unlock with Full AccessThe challenge
A team is considering fine-tuning its model before request volume grows. Common document types already perform well, while newer customer schemas account for most of the errors. Write a decision memo using the error breakdown, available corrections and production constraints to choose what the team should build next.
- The document text is already available
- OCR and layout parsing are handled upstream. Your recommendation concerns how the model understands and extracts the requested fields.
What you'll practise
Diagnosing the accuracy gap
Use the customer and document breakdown to identify where improvement is needed.
Choosing an approach
Evaluate the fine-tuning proposal against other ways to improve the extraction step.
Production economics
Account for changing schemas, growing traffic, inference cost and latency.
Your recommendation
Explain the proposed approach and the evidence behind it.
The full brief and feedback are included with Full Access.