AnthropicVerified interview challenge
Review an inference API design
Review a proposed inference service and explain how you would improve it.
Unlock with Full AccessThe challenge
A team has proposed an online API that serves model predictions at high request volume. Examine its batching, worker allocation and retry behaviour. Work through follow-up questions as traffic, model requirements and failures change.
- A written design review
- Keep the proposal beside your response throughout the interview. Use the canvas if it helps explain your changes.
What you'll practise
Review the proposal
Identify the design choices that affect the service and explain which changes matter most.
Reason about capacity
Connect request volume, batch execution and waiting time to the capacity the service needs.
Handle changing conditions
Explain how the design responds to uneven model traffic, overload and worker failures.
Your design review
Proposed APIRequests and batching
Revised designCapacity and reliability
The interview, drawing canvas and final report are included with Full Access.