Google Cloud API Gateway Ships Serverless Multi-Model Routing for Gemini and Claude

Google Cloud API Gateway just shipped model routing in Public Preview. It targets one of the more tedious parts of building multi-model AI applications: managing a separate endpoint, SDK, and payload format for every model provider you want to use.

The proxy problem it replaces

The standard workaround today is running something like LiteLLM as a self-hosted sidecar. That gives you a proxy server to deploy, scale, monitor, and keep alive. API Gateway's model routing replaces that with a serverless ingress layer that accepts OpenAI-compatible requests and dynamically routes them to Gemini, Claude, or OpenAI-compatible models on Vertex AI. No proxy infrastructure to manage. The routing logic lives in your OpenAPI spec, and the gateway handles the rest at the network edge.

How routing actually works

When a request arrives, the gateway runs four steps in sequence:

1. Intercepts the incoming POST /chat/completions request
2. Reads the model field in the JSON payload (e.g. {"model": "claude-opus-4-7"})
3. Matches that model name against routing rules defined in your OpenAPI 3.x spec, falling back to a default model if no rule matches
4. Transcodes the OpenAI-compatible payload in-flight into the native Vertex AI schema, then dispatches it to the correct backend

That last step carries the real weight. Gemini, Claude, and OpenAI-compatible models each use different request and response schemas. The gateway translates between them transparently, so your client code stays the same regardless of which model is actually serving the request.

Configuration in three steps

Everything is configured inside an OpenAPI 3.x spec using a new x-google-api-management extension block. You define named backends pointing to Vertex AI endpoints, then wire them into a router with rules and a default fallback:

x-google-api-management:
  backends:
    gemini-flash:
      address: https://aiplatform.googleapis.com/v1/projects/YOUR_PROJECT/
              locations/global/publishers/google/models/gemini-3.5-flash-lite:generateContent
    claude-opus:
      address: https://aiplatform.googleapis.com/v1/projects/YOUR_PROJECT/
              locations/global/publishers/anthropic/models/claude-opus-4-7:rawPredict
  ai:
    models:
      routing:
        routers:
          my-router:
            defaultModel:
              backend: gemini-flash
              targetModel: google/gemini-3.5-flash-lite
            rules:
              - model: "claude-opus-4-7"
                backend: claude-opus
                targetModel: anthropic/claude-opus-4-7

Once deployed, your application sends a standard OpenAI chat completions request and sets the model field to whichever backend it wants. The gateway routes it from there:

curl -X POST "https://my-gateway-url.com/v1/chat/gemini-claude" \
  -H "content-type: application/json" \
  -H "x-api-key: $API_KEY" \
  -d '{
    "model": "claude-opus-4-7",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Where it fits in your stack

API Gateway works standalone for rate limiting and token tracking, or alongside the Gemini Enterprise Agent Platform. A common pattern: route your agent's egress through Agent Gateway for security governance, then pass the request to API Gateway for dynamic model routing. The gateway becomes a single control plane for AI traffic, giving you one place to enforce auth, quotas, and observability across every model call your agents make.

What it handles well

• Standardized interface: One OpenAI-compatible endpoint for all models, with no per-provider SDK juggling
• No proxy ops: No LiteLLM or equivalent server to deploy, scale, or patch
• Centralized governance: Auth, rate limiting, and token tracking in one place for platform and security teams
• In-flight transcoding: Payload format differences between Gemini, Claude, and OpenAI-compatible models are handled automatically
• Streaming support: Server-sent events (SSE) work out of the box for streaming responses

Current limitations worth knowing

This is a Public Preview, and the constraints are real. Before you commit:

• Same-host only: All backends in a single router must share the same hostname (e.g. aiplatform.googleapis.com). Cross-host routing within one router is not supported.
• Text only: Model routing currently handles text-based JSON payloads. Multimodal routing is not yet available.
• No VPC Service Controls: Gateways with model routing enabled cannot sit inside VPC Service Controls perimeters, which may block adoption in regulated industries.
• No mixed configs: An OpenAPI spec must be entirely model-routing or entirely standard routing. The two cannot coexist in the same spec.
• No in-place migration: Existing gateways cannot be retrofitted with model routing. You need a new API config and gateway instance.
• Silent misrouting: If a client omits the model field, the gateway misroutes the request without returning an error.

Pricing and availability

The first 2 million API Gateway calls per month are free, then $3.00 per million up to 1 billion calls. Model inference through Vertex AI is billed separately. For most teams, the gateway cost will be negligible compared to token spend.

Model routing is available now in Public Preview. It requires an OpenAPI 3.x spec (Swagger 2.0 is not supported), and models must be served through Vertex AI Model Garden's Model as a Service offering.

How it compares to existing tools

OpenRouter, LiteLLM, and TrueFoundry's AI Gateway have been solving multi-model routing for a while, and they support far more providers. Google's offering differs in character: it is managed infrastructure, deeply integrated with Vertex AI's security and governance surface, with no sidecar to operate. For teams already on Google Cloud who want to skip running their own proxy layer, that tradeoff is straightforward. Configure a YAML spec, skip the ops burden.