OpenAIBased on real interviews
Spot the bugs: ring all-gather
Trace a ring all-gather implementation and find why ranks do not finish with the correct ordered chunks.
Unlock with Full AccessThe challenge
Each rank begins with one chunk of data. After the collective completes, every rank should hold all chunks in order. Read the implementation and follow how data moves around the ring, then explain the code faults, their effect on the output and the required fixes.
What you'll practise
Rank and chunk tracking
Follow which data each rank holds as the collective progresses.
Communication steps
Compare the implementation's transfers with the expected ring behaviour.
Output correctness
Connect each fault to missing, repeated or incorrectly placed data.
Ring all-gather implementationCode included in the full challenge
Your diagnosis
Explain how the faulty data movement changes the result.
The code, instructions and feedback are included with Full Access.