LLM inferenceRapid Fire
LLM Inference Fundamentals 1
Answer seven questions about the memory and computation behind LLM inference.
Unlock with Full AccessThe round
Explain how an inference server processes a prompt, generates tokens and reuses earlier computation. The round covers KV caches, prefill and decoding, memory use and techniques for improving generation speed. Some questions ask you to reason through a change in model or workload size.
What you'll practise
Inference phases
Distinguish the work done before the first token from the work repeated during generation.
Cache memory
Reason about how model architecture and active requests affect memory requirements.
Faster generation
Explain how cache reuse and speculative decoding change the work an inference system performs.
Inference fundamentals
Answer seven questions
Get feedback on each answer when you finish the round.
All questions and feedback are included with Full Access.