Loss and entropyRapid Fire
Loss & Entropy Fundamentals
Practise the maths behind cross-entropy, masking, label smoothing and KL divergence.
Unlock with Full AccessThe round
These questions connect loss functions to the probability distributions and numerical calculations underneath them. Work with the conventions stated in each question, from averaging selected token losses to handling shifted exponential sums. Explain why the relationships hold as well as writing the equations.
What you'll practise
Loss calculations
Reason about cross-entropy, masked averages and smoothed targets.
Distribution relationships
Connect entropy, cross-entropy and KL divergence, including support and direction.
Numerical stability
Keep exponential calculations consistent when the reference value changes.
Loss & entropy
Answer seven questions
Get feedback on each answer when you finish the round.
All questions and feedback are included with Full Access.