GitHub's Copilot Code Review Got 20% Cheaper by Rewriting Instructions, Not Tools
GitHub's Copilot code review got worse after a tool upgrade — until they rewrote the instructions, cutting costs 20% with no quality loss
- GitHub migrated Copilot code review to shared CLI tools (
grep,glob,view) expecting an upgrade, but benchmarks showed higher costs and fewer issues caught. - The root cause was tool instructions written for a general coding assistant, not a reviewer — causing the agent to browse repos instead of investigating diffs.
- Rewriting instructions to encode a reviewer's workflow (ask, narrow, read, decide) fixed the regression without changing the tools at all.
- The result: ~20% lower average review cost in both offline and production evaluation, with no quality degradation.
- Key insight: for agents, tool instructions are as load-bearing as API docs — a small wording change can reshape the entire investigation pattern.
- The improvements are live now; the 4,000-character limit on
copilot-instructions.mdfiles has also been removed for more customization flexibility.
There's a widely-held assumption in agentic AI development: give the agent better tools, and it will do better work. GitHub just published a detailed post-mortem showing exactly how that assumption burned them , and what they learned fixing it.
When you open a pull request, Copilot code review reads the diff and explores the surrounding code to find problems before they ship. To do that, it used its own set of code exploration tools , until the team swapped in the better-maintained, shared tools that power the Copilot CLI: grep, glob, and view. It seemed like a clean infrastructure win.
Instead, benchmarks showed that the cost of reviews went up and fewer issues were being caught. But the tools weren't the problem. The instructions were. Once GitHub rewrote them to match how a reviewer actually reads a pull request, the regression flipped into a win: roughly 20% lower average review cost, while maintaining the same review quality.
Why the migration made things worse
The original Copilot code review tools were inspired by earlier agentic systems like SWE-agent-style repository navigation and GitHub Copilot Autofix. Those tools were specific to code review and designed for how models behaved at the time , earlier agentic models made fewer tool calls and were worse at automatically pulling in context, so it was more important to include all relevant information in the few calls that were made.
The Copilot CLI harness is also used by a growing number of Copilot agent products, including the GitHub Copilot cloud agent, so harness improvements can benefit more than one product. GitHub wanted to clean up and share infrastructure, so they experimented with using the CLI tools in code review , the goal being to reduce duplicated implementations and make it easier to carry improvements across Copilot products.
The tool mapping looked straightforward on paper:
| Old Copilot Code Review | Copilot CLI | Purpose |
|---|---|---|
list_dir | glob | Discover candidate files and directories |
search_file / search_dir |