Anthropic
OpenAI
Google DeepMind
MetaBased on real interviews
Plan a multi-region inference-serving rollout
Expand an inference service into new regions while meeting a demanding failover target on a fixed GPU budget.
Unlock with Full AccessThe challenge
An LLM service runs in one US region and must expand to Europe and Asia within two quarters. Its GPU budget is fixed, regional hardware availability differs, and a content-policy dependency is expected to remain in the original region.
Write the design memo that makes the rollout and reliability commitments credible. Work through regional capacity, model distribution and request routing, then explain how the service behaves when a region fails and which operating assumptions the team still needs to settle.
What you'll practise
Regional dependencies
Decide where components run and explain which cross-region dependencies remain.
Capacity and model delivery
Account for GPU availability, warm capacity and the cost of moving a large weight set.
Operating through failures
Trace specific failures from detection to recovery and define the team's responsibilities.
Explain the architecture, capacity assumptions and recovery plan.