Google Cloud API Gateway Ships Serverless Multi-Model Routing for Gemini and Claude
Google Cloud API Gateway now routes traffic across Gemini, Claude, and OSS models from a single OpenAI-compatible endpoint, no proxy servers required.
- Google Cloud API Gateway model routing is now in Public Preview, routing Gemini, Claude, and OpenAI OSS models from one endpoint.
- Routing logic lives in an OpenAPI 3.x spec using a new
x-google-api-managementextension block, no proxy server needed. - The gateway transcodes OpenAI-compatible payloads in-flight to each provider's native Vertex AI schema automatically.
- Key limits: same-host backends only, text modality only, no VPC Service Controls, no mixed routing configs in one spec.
- Pricing: first 2M gateway calls/month free, then $3.00/million; model inference billed separately through Vertex AI.
- Can be used standalone or chained with the Gemini Enterprise Agent Platform for full security governance.
Google Cloud API Gateway just shipped model routing in Public Preview, and it solves one of the most tedious parts of building multi-model AI applications: managing a different endpoint, SDK, and payload format for every model provider you want to use.
The proxy problem it replaces
When building AI applications, developers need the freedom to route traffic to the best model for the job without hardcoding endpoints or managing open-source proxies. The usual workaround is running something like LiteLLM as a self-hosted sidecar, which means you now have a proxy server to deploy, scale, monitor, and keep alive. Google Cloud API Gateway now offers model routing in Public Preview to solve this, providing a lightweight, serverless ingress layer that accepts OpenAI-compatible requests and dynamically routes them to Gemini, Claude, or OpenAI OSS-GPT.
The key word is serverless. There is no proxy infrastructure to manage. The routing logic lives in your OpenAPI spec, and the gateway handles the rest at the network edge.
How it works under the hood
The architecture is straightforward. When a request comes in, the gateway does four things in sequence:
- Intercepts the incoming
POST /chat/completionsrequest - Inspects the
modelfield in the JSON payload (e.g.{"model": "claude-opus-4-7"}) - Matches that model name against routing rules you defined in your OpenAPI 3.x spec, falling back to a default model if no rule matches
- Transcodes the OpenAI-compatible payload in-flight into the native Vertex AI schema, then dispatches it to the right backend
That last step is the real work. Each provider (Gemini, Claude, OpenAI OSS) has a different request/response schema. The gateway handles that translation transparently, so your client code never changes regardless of which model is actually serving the request.