Verified interview challenge
Reassess an enterprise coding-agent pilot
Recommend what to change when engineers use a coding agent heavily but their delivery metrics get worse.
Unlock with Full AccessThe challenge
A coding-agent pilot has high daily use and healthy suggestion acceptance, yet the participating teams are merging fewer pull requests. The customer is nearing its expansion decision, and a skeptical platform leader questions whether the tool helps this codebase.
Use the available IDE activity and engineering workflow evidence to investigate the conflicting signals. Your recommendation must give the customer a credible next step within the remaining pilot window, without assuming that more usage proves productivity or that every code change can be attributed to the agent.
What you'll practise
Causal diagnosis
Use cohort and workflow evidence to distinguish explanations for the throughput drop.
Pilot decisions
Choose changes that can produce useful evidence before the gate.
Commercial communication
Explain the recommendation to stakeholders with different incentives and concerns.
Write your reasoning and submit it for feedback.