Skip to main content
Back

When a Small Model Should Ask for Help (LLM Cascades)

LLM cascades let a small model like DeepSeek V4.1 Flash handle work alone, escalating only 22% of cases to Claude Opus 5.5 while cutting cost per 1,000 cases from $3.86 to $0.88

  • In a 4 November 2026 identity-grading test, Jev routed 22% of uncertain cases to Claude Opus 5.5, scoring 98.8% at $0.88 per 1,000 cases versus 99.0% at $3.86 for Opus alone.
  • A Carnegie Mellon paper (29 October) found its small judge's confidence-accuracy separation score fell to 0.498 on reference-free prose, versus 0.84 on same-style answers, and trailed GPT-6 by 12.9 points on code, 14.3 on math, and 27.6 on logic puzzles.
  • Across three studies, confidence-based cascades beat random case selection by only small margins, such as 90.2% versus 89.0% at matched spend in the Carnegie Mellon analysis, and underperformed random routing in a 31 October serving paper when the stronger model cost 2 to 3 times as much.
  • In a 3 November 2026 experiment logging 1,188 agent runs across 12 hard coding tasks, Claude Opus 5.5 alone passed 81% of tasks at $1.20 each, while a team pairing Opus with DeepSeek V4.1 Flash 1017 passed only 67% at $1.29, with 71% of team failures traced to the small worker's own errors.
  • On a bug-fixing benchmark, DeepSeek V4.1 Flash 1017 fixed 42.7 of 105 planted bugs on average in October, roughly matching Claude Opus 5.5's 41.7 of 105 and well above Claude Opus 5's 27 scored in August.

Over the past few months, most of us have handed almost every task to a frontier model. When smaller models joined in, they came in as workers: the frontier model wrote the plan, passed the pieces down, and reviewed what came back.

But this article looks at the opposite arrangement. The small model does the work on its own and asks the frontier model for help only when it needs it, so the hard part becomes deciding when to ask.

By small model I mean the cheaper, faster tier a lab ships, such as DeepSeek V4.1 Flash 1017, Claude Haiku 5.5, or OpenAI's GPT-6 Luna.

And on some work, the small tier now scores where frontier models did a few months ago. Our DeepSeek architecture piece explains why Sebastian Raschka called the release a “big overhaul” that DeepSeek could have named V5. And also, I tested it, and it scored as the frontier open model that was Kimi K3 back then.

Also, on one engineer's bug-fixing benchmark, DeepSeek V4.1 Flash 1017 fixed 42.7 of 105 planted bugs on average in October, level with Claude Fable 5.1's 43 in September and well above Claude Opus 5's 27 in August.

Running the small model first and sending only the hard cases up is often called a cascade. Whatever decides which cases go up is the handoff rule.

My answer is that the handoff pays only when that rule passes three questions:

  • Does the rule reach the small model's mistakes?
  • Does it beat sending the same number of cases up at random?
  • Can the frontier model fix what it receives?

The rule can be the small model asking for help itself (as in the advisor tool Anthropic described in April 2026), a score that lands in a set range, or a check that fails, such as a test.

The saving can be large. On 4 November 2026, a customer-data company had Jev(a decision model), grade synthetic identity records and send the 22% of cases it was unsure about to Claude Opus 5.5.

On the easier of its two sets, the pair scored 98.8% at $0.88 per 1,000 cases, against 99.0% and $3.86 for Opus alone. It never compared that 22% with a random 22%, so the saving may come from sending fewer cases up.

Picture the setup as a line. A request reaches the small model, the handoff rule reads its answer, and the case either stays there or goes up to the frontier model.

Questions 1 and 2 test the rule, and question 3 tests what happens after the handoff.

Each of the next three sections takes one question, answers it first, then shows the evidence and what to do. Section 1 has the case where the more accurate small model made the worse partner, section 2 shows what a random pick does to the saving, and section 3 shows why a small-coder team scored below Opus alone.

Section 4 applies all three questions to coding agents, where they are hardest, and section 5 turns them into six steps you can run on your own setup.

1. Does your handoff rule reach the small model's mistakes?

If the rule reads the small model's own confidence, it reaches only the mistakes the model is unsure about. The ones it makes with full confidence stay in the output, because nothing marks them for a second look.

That follows from how a score-based rule works. It reads a confidence score, the number a model gives with its answer to say how sure it is, then keeps the confident answers and sends the uncertain ones up.

That 4 November grading test from the opening also tried OpenAI's GPT-6 Luna as the small model. On its own, Luna was more accurate than Jev, at 97.3% against 95.6% on the easier set.

Then they put Claude Opus 5.5 behind each of them on the harder set, and the order flipped, the Luna pairing scored 91.6% and the Jev pairing 95.3%.

Luna's mistakes sat at scores of 90% to 100%, above the range the rule sends up, so Opus never saw them.

The same weakness shows up in a Carnegie Mellon paper (29 October) on a small judge, a model that grades other models' answers. The authors measured how well the judge's confidence separates its right verdicts from its wrong ones, on a scale where 0.5 is a coin flip, 1.0 means every right verdict gets a higher confidence than every wrong one.

That measure was 0.84 when the right and wrong answers were written in the same style, 0.76 when the wrong answer was the more polished one, and 0.498 on prose with no reference answer (the confidence said almost nothing about whether a verdict was right).