Gemini 3.6 Flash Debugging Benchmark: It Can Fix Real Bugs, but Is It Efficient?

We tested Gemini 3.6 Flash on 13 debugging tasks and compared its accuracy, cost, speed, and token usage with eight other models.

Google introduced Gemini 3.6 Flash as a workhorse model built for faster, more efficient agentic workflows. The company says it uses 17% fewer output tokens than Gemini 3.5 Flash on the Artificial Analysis Index, takes fewer reasoning steps and tool calls, and lowers the output price from $9 to $7.50 per million tokens.

Those improvements sound useful for coding agents. But a cheaper token is not automatically a cheaper bug fix.

When an AI model debugs an existing codebase, the final patch is only one part of the job. The agent must inspect unfamiliar code, follow data across files, form and test hypotheses, run commands, interpret failures, and sometimes repeat the entire loop. A model can have a low API price and still become expensive if it needs a large amount of context or takes a long path to the answer.

That is what we wanted to measure.

We ran Gemini 3.6 Flash against 13 private debugging tasks in an existing full-stack application. We repeated each task five times, then compared its attempts with eight other models running the same benchmark.

The model resolved most of the attempts and performed especially well when a bug required tracing behavior across multiple parts of the application. Its patches were generally small and focused.

However, it finished in the lower half of the comparison, and the resources required to produce those patches complicated Google's efficiency story. The more important question is what it costs to get there.

What Google claims about Gemini 3.6 Flash

Google positions Gemini 3.6 Flash as an improvement over Gemini 3.5 Flash in both quality and efficiency.

The company reports that the new model:

• Uses 17% fewer output tokens on the Artificial Analysis Index
• Takes fewer reasoning steps and tool calls in multi-step workflows
• Costs $1.50 per million input tokens and $7.50 per million output tokens
• Improves from 37% to 49% on DeepSWE
• Improves from 55.1% to 58.7% on SWE-Bench Pro
• Improves from 76.2% to 78% on Terminal-Bench 2.1

Google also says the model produces fewer unwanted code edits and shorter execution loops than Gemini 3.5 Flash. These comparisons are all against its predecessor, which matters when interpreting our results.

Google's official Gemini 3.6 Flash comparison table showing pricing and results on SWE-Bench Pro, DeepSWE, Terminal-Bench, MLE-Bench, OSWorld, and long-context benchmarks.

Google's own comparison does not present Gemini 3.6 Flash as the strongest coding model overall. It trails GPT-5.6 Luna, Grok 4.5, and Claude Sonnet 5 on SWE-Bench Pro, DeepSWE, and Terminal-Bench. It performs better on other workloads, including long-context evaluation, computer use, and some forms of chart reasoning.

Gemini 3.6 Flash performance comparison against other frontier models.

Our benchmark focuses on a much narrower question: Can Gemini 3.6 Flash take a bug report, investigate an existing codebase, produce a patch, and pass tests it has never seen?

How the benchmark works

This is not a collection of isolated code-completion questions.

The benchmark places a coding agent inside SignalDesk, a private newsletter operations application with roughly 7,000 lines of Python and TypeScript. The application includes a FastAPI data pipeline and a Next.js frontend.

Each task starts from a deliberately broken version of the repository. The bugs resemble issues that might arrive from an editor, operations engineer, analyst, or security reviewer.

For example, one report describes morning stories appearing under the wrong day for readers in Manila and Sydney. Another describes an ingestion job that still produces correct results but now takes 25 minutes. A third describes an admin token that accepts any matching prefix instead of requiring the complete token.

The model does not receive the faulty line or the expected patch. It receives a symptom-focused bug report and must find the root cause itself.

Inside the benchmark, the model can:

• Browse and search the repository
• Open and edit files
• Run shell commands
• Execute relevant tests
• Inspect test failures
• Iterate until it believes the bug is fixed

It works inside a fresh Docker sandbox with no internet connection. This matters because the model cannot search for the repository, copy an answer from an issue tracker, or rely on external documentation while solving the task.

The agent is instructed to make the smallest correct change, avoid modifying tests, run the relevant test suite, and submit a short explanation when finished.

Benchmark workflow diagram showing the bug report entering the coding agent, investigation inside a Docker sandbox, code changes, visible tests, hidden tests, the full regression suite, and the final resolved or failed result.

Passing the tests available to the agent was not enough. After each submission, we injected hidden tests and ran the full Python and TypeScript regression suites. We counted an attempt as resolved only if the visible tests, hidden tests, and regression tests all passed and the model had not modified any test files.