AnthropicVerified interview challenge
Concurrent crawler bug hunt
Find the correctness problems introduced when a web crawler starts using several worker threads.
Unlock with Full AccessThe challenge
The crawler should visit each page on the seed host once and finish when its work is done. Inspect the threaded implementation for paths that skip or repeat pages, leave workers hanging or send requests outside that host. Explain what causes each failure and how you would fix it.
What you'll practise
Shared worker state
Reason about how several threads discover, claim and process the same work.
URL identity
Check which URLs represent the same page and which are outside the allowed host.
Crawl completion
Identify execution paths that lose work or prevent the crawl from finishing.
Threaded crawlerCode included in the full challenge
Your diagnosis
Trace the failure and describe the correction for each bug.
The code, instructions and feedback are included with Full Access.