Numerical precisionRapid Fire
Numerical Precision Fundamentals
Explain the trade-offs behind low-precision training, from floating-point formats to scaling and quantization.
Unlock with Full AccessThe round
Work through short questions on FP16, BF16, FP8 and INT8. The round connects what each format can represent to the scaling and parameter-state choices used in training. You will explain mechanisms and perform calculations under the conventions given in each question.
What you'll practise
Range and precision
Compare number formats and explain what their bit layouts mean for stored values.
Training stability
Reason about loss scaling, overflow and higher-precision parameter state.
Quantization scales
Calculate and interpret scales, including the effect of values outside the calibrated range.
Numerical precision
Answer seven questions
Get feedback on each answer when you finish the round.
All questions and feedback are included with Full Access.