Databricks' Smart Routing Cuts AI Coding Costs by 56% Without Sacrificing Quality

Every time a developer fires up Claude Code or Codex and leaves the model selector on the most capable (and most expensive) option, they're probably overpaying. Databricks found that models cluster into capability tiers, and that much everyday work , flipping a flag, a single-file edit, a well-scoped bug fix , simply does not require the most expensive models. The fix sounds obvious: route each task to the cheapest model that can handle it. The hard part is doing that automatically, without making developers think about it. That's exactly what Smart Routing does.

Databricks is launching Smart Routing as the next major cost control in Unity AI Gateway, now available in Beta. Unity AI Gateway provides a central place to get access to AI, manage spend, and enforce controls across an enterprise, and Smart Routing adds intelligent optimization by automatically matching tasks to the right model based on complexity. Smart Routing works directly in Claude Code and Codex, allowing teams to optimize the tools developers already use.

The cost problem hiding in plain sight

The price and performance frontier for coding tasks features a huge diversity of models and harnesses , in 2026 alone, 33 new models have been released. That explosion of choice has a paradoxical effect: one of the biggest opportunities is matching each task to the right model instead of defaulting every task to the most capable option. Just leveraging lower-cost models can save 50%+, but it's incredibly daunting for users. Faced with choice overload, most teams just pin the most powerful model at maximum effort and move on , paying frontier prices for tasks that don't need frontier intelligence.

Unity AI Gateway governs what agents do at runtime: hard spend caps, smart routing, service policies, and PII guardrails. Smart Routing is the newest addition to that stack, and it's the one that directly attacks the token bill.

How the router actually works

Smart Routing uses task-aware routing , meaning it makes one routing decision per session at the start, rather than re-routing every individual message. This design choice matters for a non-obvious reason: at scale, AI costs are dominated by cache hit rate. Routing consecutive turns to the same model keeps the cache warm; constantly switching models would blow it up and erase most of the savings.

The routing pipeline has two steps:

1. Task classification: A small, cheap, low-latency model reads the task description and labels it across several semantic dimensions , which part of the system changes, what code evidence the prompt contains (a snippet, a traceback, or nothing), how localized the fix looks, and what kind of project it belongs to. From these labels, the router derives a task-type family and a language family.
2. Model selection: The router defaults to a mid-tier model and uses those labels to escalate up (to a frontier model for complex, cross-system work) or delegate down (to a cheaper model for simple, well-scoped tasks). A single routing policy covers the entire model suite.

Critically, the router uses only information available at the start of the task , description and metadata. It doesn't need the answer, tests, or anything about the repository, and Smart Routing is still able to choose effectively.

Beyond model routing: harnesses matter too

Choosing the right model is only half the equation. Different coding harnesses , Claude Code, Codex, custom agents , have different strengths, and the best outcome often comes from pairing the right model with the right harness. That's where Omnigent comes in.

Omnigent is Databricks' open-source meta-harness , a layer that sits above individual coding agents and makes them interoperable. When Smart Routing is enabled inside Omnigent, it selects both the harness and the model automatically for each task. This also unlocks a powerful pattern for multi-agent work: a single task can experience nuanced routing decisions across planning and parallel sub-agent work , for example, routing large codebase summarization tasks to cheaper models while designing the architecture with more expensive ones , leading to even more substantial savings.

The numbers

Databricks tested Smart Routing on two benchmarks:

• Internal benchmark (their own multi-million-line codebase, which no external labs have seen): Smart Routing outperformed any single model at 65% of the cost of a leading frontier model like Opus 5 , a 35% cost reduction.
• Public coding benchmarks: Smart Routing matched Opus 5 on performance at less than half the cost , a 56% cost reduction.

The internal benchmark is the more meaningful one. Public benchmarks feature unusually clean, self-contained tasks that are easy to route. Real developer sessions are messier: opening prompts are vague, sessions get reused across multiple tasks, and complexity evolves mid-conversation.

Where it struggles today

The team is candid about the current limitations. The router commits to a model on the first message of a session , which is often the worst possible moment to decide, since opening prompts are rarely precise specifications. Three specific failure modes stand out:

• Vague opening prompts: A developer types a symptom or rough intention, not a spec. The router reads that and commits.
• Session reuse: The routing decision that was right for the first task in a session can be wrong by the fourth, and nothing re-triggers the router.
• Mid-session switching costs: Switching models mid-session is expensive because it causes a cache miss, which currently makes it impractical at scale.

What's coming next

Databricks has a clear research roadmap to close these gaps:

• Route after a few turns, not the first one. Let a cheap model handle the initial back-and-forth and clarifying questions, then route once the task has taken shape.
• Start with fully-specified tasks. PR reviews, batch migrations, and scheduled jobs arrive with machine-written task statements , perfect routing inputs. This is where they're deploying first.
• Make sessions smaller. Sessions that stay on one task route better and cost less. Tooling can encourage fresh sessions when the subject changes.
• Make mid-session switching affordable. Context compaction events (where a cache miss is already happening) are the natural seam for model switches , similar to how Cognition's Devin Fusion handles it.

The bigger shift

Unity AI Gateway is now Generally Available, giving enterprises a unified way to govern AI spend, security, and access across agents, models, MCPs, skills, and tools. As AI adoption scales, token-based costs can grow quickly and unpredictably. Unity AI Gateway provides end-to-end observability, cost controls, and smart routing to maximize the value of every AI dollar.

The framing Databricks uses is worth noting: they call it moving from "tokenmaxxing" to "valuemaxxing." Coding tools today often incentivize consuming more tokens. What actually matters is productive output per dollar. A faster, cheaper model that solves the task is strictly better than a slower, expensive one that also solves it , and keeping frontier capacity free for tasks that genuinely need it has compounding benefits across a large engineering team.

To get started: enable Smart Routing through Unity AI Gateway for model-only routing, or use Omnigent v0.9.0+ for combined model and harness routing.