There's a widely-held assumption in agentic AI development: give the agent better tools, and it will do better work. GitHub just published a detailed post-mortem showing exactly how that assumption burned them , and what they learned fixing it.
When you open a pull request, Copilot code review reads the diff and explores the surrounding code to find problems before they ship. To do that, it used its own set of code exploration tools , until the team swapped in the better-maintained, shared tools that power the Copilot CLI: grep, glob, and view. It seemed like a clean infrastructure win.
Instead, benchmarks showed that the cost of reviews went up and fewer issues were being caught. But the tools weren't the problem. The instructions were. Once GitHub rewrote them to match how a reviewer actually reads a pull request, the regression flipped into a win: roughly 20% lower average review cost, while maintaining the same review quality.
Why the migration made things worse
The original Copilot code review tools were inspired by earlier agentic systems like SWE-agent-style repository navigation and GitHub Copilot Autofix. Those tools were specific to code review and designed for how models behaved at the time , earlier agentic models made fewer tool calls and were worse at automatically pulling in context, so it was more important to include all relevant information in the few calls that were made.
The Copilot CLI harness is also used by a growing number of Copilot agent products, including the GitHub Copilot cloud agent, so harness improvements can benefit more than one product. GitHub wanted to clean up and share infrastructure, so they experimented with using the CLI tools in code review , the goal being to reduce duplicated implementations and make it easier to carry improvements across Copilot products.
The tool mapping looked straightforward on paper:
| Old Copilot Code Review | Copilot CLI | Purpose |
|---|---|---|
list_dir | glob | Discover candidate files and directories |
search_file / search_dir |
Don't miss what's next in AI
Join 300,000+ engineers and researchers who get the signal, not the noise.
- Full access to in-depth AI research breakdowns
- Be the first to know what's trending before it hits mainstream
- Daily curated papers, repos, and industry moves