Databricks' Unity AI Gateway Now Governs Every Agent Touching Your Data

Enterprise AI has a sprawl problem. Most large organizations are no longer running one or two models in production. They're managing fleets of coding agents, MCP servers, custom bots, and third-party AI apps, all pulling from sensitive data and racking up token bills with no central oversight. Databricks' Unity AI Gateway is now generally available, positioning itself as the single control plane to govern all of it.

The sprawl that forced this

AI deployments have become multi-model, multi-agent, and multi-vendor almost overnight. Developers run coding agents. Business users query enterprise data through AI interfaces. Internal teams launch custom agents to automate workflows. The result is what the industry calls "agent sprawl," and it creates three compounding problems:

• Runaway costs: Consumption-based token pricing means spend scales with usage, not headcount. Most AI tools have no native cost cap, and no single view of the full bill across vendors.
• Security and IP risk: Agents need access to sensitive data to be useful, which exposes PII, creates prompt injection vulnerabilities, and leaves confidential information sitting in agent traces that can be inadvertently stored or leaked.
• Vendor lock-in: Managing thousands of custom agents across disconnected platforms makes swapping models or adopting new tools nearly impossible without rebuilding governance from scratch.

What Unity AI Gateway actually does

Unity AI Gateway is a centralized runtime governance layer for AI agents, built as an extension of Unity Catalog. Traditional governance answers "who can access this asset." The Gateway answers a different question: what is an AI system permitted to do during a live interaction. That distinction matters because it shifts control from access-time to runtime, covering behavior, not just permissions.

The core capabilities fall into three areas:

• Cost observability and control: Track token consumption, enforce budgets, and apply rate limits and traffic controls across users, teams, applications, and model providers. All data lands in Unity Catalog, where built-in dashboards surface spend immediately.
• Runtime guardrails: Enforce centralized policies with identity-aware controls. Capture prompts, traces, tool calls, payload logs, audit logs, and policy decisions across every AI interaction.
• Open multi-model access: Access GPT, Claude, Gemini, and other models through Foundation Model APIs on a pay-per-token basis, with no infrastructure to manage. External model providers can also connect and route through the same access control and traffic management layer.

Smart Routing: powerful, still in beta

Instead of sending every request to the most expensive frontier model by default, Smart Routing dynamically directs each call based on quality requirements, cost, latency, and available budget. Hard tasks go to GPT-4-class models; simpler completions route to cheaper or open-source alternatives automatically. The feature is still in beta, and teams need to contact their Databricks account team to get access.

MCP governance: closing a real attack surface

Model Context Protocol (MCP) is the emerging standard for giving agents access to enterprise tools such as databases, APIs, and internal applications. Each MCP provider typically ships with its own credentials, OAuth app registrations, and token refresh logic. Without a centralized layer, different teams end up managing client secrets, rotation schedules, and per-tool auth flows in isolation.

Unity AI Gateway collapses that into a single identity layer backed by Unity Catalog permissions. Agents inherit user permissions automatically, access is limited to approved tools, and every action produces an audit-ready log.

Production scale before the announcement

Over a quadrillion tokens have passed through Unity AI Gateway in the past year across thousands of customers, including Rivian, Asana, and Edmunds. Zepto reported handling over 100 billion tokens per month through the gateway with no availability issues. Databricks also uses the product internally to manage AI spend across thousands of employees with access to multiple AI tools.

Who gains, who feels pressure

Platform teams and security or compliance teams inside enterprises already on Databricks get the clearest benefit: a governance layer that previously required stitching together multiple tools. Developers stop managing separate credentials and auth flows for each AI tool they use.

The competitive pressure lands on standalone AI gateway vendors. Portkey and Helicone lead on AI-specific observability. LiteLLM wins on self-hosted flexibility. Neither has Databricks' data governance foundation or the Unity Catalog integration that makes policy enforcement coherent across data and AI in one place. Snowflake and Palantir are also building AI governance solutions, so Databricks is moving early to establish the standard.

What opens up from here

Databricks has launched a Unity AI Gateway partner ecosystem with integrations across AI security, identity governance, agent discovery, data protection, and threat detection. Existing enterprise security tools, including identity providers and SIEM platforms, can plug into the governance layer without replacement.

Unity Catalog now underpins governance for over 14,000 organizations. The Gateway is its runtime enforcement arm: Unity Catalog defines what exists and who can access it; the Gateway controls what agents are actually allowed to do while running.

For teams already on Databricks, documentation is live on AWS, Azure, and GCP. An AI governance webinar with Databricks co-founder and CTO Matei Zaharia is scheduled for August 13. For teams not on Databricks, the GA signals that the AI gateway space is consolidating fast around platforms that tie governance to data lineage, and the window for standalone tools to carve out durable differentiation is shrinking.