DeepSeek V4 Flash Debugging Benchmark: Can It Match Fable 5 at 1/99th the Cost?

DeepSeek V4 Flash has been making a lot of noise in the AI community since it launched a few days ago. Much of the attention comes from DeepSeek’s promise of coding performance that can compete with frontier models at a fraction of the price.

Former Meta senior PM Xiaoyin Qu went even further by predicting that DeepSeek V4 Flash could become the biggest market-share winner by pulling developers away from Claude. Yes, that’s a bold claim for a smaller, open-weight model competing with Claude.

We at AlphaSignal wanted to know whether those claims would hold up when the model had to debug a private repository instead of answering public benchmark questions. So we gave DeepSeek V4 Flash 18 bugs, ran each task three times, and graded every patch against hidden tests and the full regression suite.

DeepSeek V4 Flash passed all 54 attempts.

The total bill was even more surprising. DeepSeek V4 Flash cost $0.21, while Fable 5 cost $20.99 for the same 54/54 score. In our benchmark, the public “around 100 times cheaper” claim worked out to 98.8 times.

DeepSeek V4 Flash was not the only model with a perfect score, and it was not the fastest. Six models went 54/54, while Grok 4.5, GPT-5.6 Sol, and Inkling finished sooner. V4 Flash ranked first because no other model combined a perfect score with a lower cost per fix.

The harder question is where that bargain stops working. What kinds of bugs would make developers better off upgrading to a more capable model?

That is why this benchmark measures the full debugging workflow. The model must understand an unfamiliar repository, find the root cause, make a safe change, and pass tests it cannot see.

V4 Flash passed every visible test, hidden test, and regression suite. But the logs revealed a more useful boundary: its advantage narrowed when the investigation became ambiguous, the patch scope expanded, or tests could no longer judge whether the solution was maintainable.

What DeepSeek claims about V4 Flash

DeepSeek V4 Flash is the smaller model in the V4 family. According to its official model card, it has 284 billion total parameters, activates 13 billion per token, supports a one-million-token context window, and ships under the MIT license.

Here’s how it compares to other V4 models:

Developers can run it in Non-think, Think High, or Think Max mode. DeepSeek recommends High for complex problem-solving and planning, while Max gives the model a larger reasoning budget for harder tasks.

DeepSeek now makes a different claim for the July 31 release. Its updated model card says V4 Flash 0731 outperforms V4 Pro Preview on all nine agent benchmarks it lists, including Terminal Bench 2.1, DeepSWE, and NL2Repo. The earlier 6.7-point Terminal Bench 2.0 and 26.9-point BrowseComp gaps came from the preview-era model card, and we did not measure those gaps in this benchmark.

However, these are DeepSeek’s own results, not an independent comparison. The public coding benchmarks used max reasoning effort with DeepSeek’s Harness minimal mode, which had not been released at the time, while two DSBench tests are internal. The comparison is also against V4 Pro Preview, not the final V4 Pro, which DeepSeek says will be released later.

Qu’s post made three claims we could compare with our logs: V4 Flash was around 100 times cheaper than premium coding models, two to three times faster, and unusually efficient when serving cached context.

The cost claim held up against Fable 5, but the speed claim did not fully carry over. V4 Flash was about 1.4 times faster than Fable in our run, not two to three times faster. Grok 4.5, GPT-5.6 Sol, and Inkling also finished sooner than V4 Flash.

How the debugging benchmark works

We ran the benchmark inside SignalDesk, a private newsletter operations app with roughly 7,000 lines of Python and TypeScript across a FastAPI data pipeline and a Next.js frontend.