Cloudflare's Clef-Omni Brings Audio and Video Into One Decision Model

Cloudflare adds a Qwen3-Omni mixture-of-experts variant that scores audio and video alongside text and images in one pass, with no token generation.

·
·
·
Cloudflare's Clef-Omni Brings Audio and Video Into One Decision ModelPRO
  • Clef-omni adds audio and video input to Cloudflare's open-weight decision model family, Apache 2.0 licensed.
  • 30B mixture-of-experts (3B active) post-trained from Qwen3-Omni-30B-A3B-Instruct, served at $0.15 per million input tokens.
  • Returns calibrated probabilities over typed questions in one forward pass, no text generation or output parsing.
  • Clef-flash price cut to $0.038/M tokens (below Jev's $0.042), but hosted context shrinks from 64K to 24K.
  • Clef is now up to 2x faster thanks to serving-layer optimizations, including a move to SGLang.
  • Weak on multi-hop reasoning (BBH, MMLU-Pro) and hallucination detection; strong on intent classification and multimodal routing.

Cloudflare brings audio and video to Clef’s typed decisions

Cloudflare has expanded its Clef decision-model family with Clef-omni weights, an Apache 2.0 model that accepts audio, video, images, and text in one request. The company also says it roughly doubled inference speed for the existing Clef model and cut Clef-flash’s input price below TypeSafe’s Jev.

Clef models turn an application state and a schema of typed questions into probability distributions for every allowed answer. One model execution can classify content, score risk, route a request, or trigger a guardrail. Calibration aims to make those probabilities operationally useful: predictions assigned 80% confidence should be correct about 80% of the time over a representative sample.

Inside the 30B model

Clef-omni uses Qwen3-Omni-30B-A3B-Instruct as its backbone. The mixture-of-experts architecture contains 30 billion parameters but activates about 3 billion for each token, reducing the computation required for inference. Cloudflare freezes that backbone and trains low-rank adapters plus a scoring head, using label-smoothed cross-entropy and a Brier penalty to improve probability calibration.

The joint schema head reads the backbone’s final hidden states, associates relevant evidence with each question, and produces one raw score for every allowed option. A per-question softmax converts those scores into probabilities. Applications receive structured decisions directly, eliminating generated-JSON parsing and retries caused by malformed output.

Deployment detail Clef-omni specification
Hosted platform Workers AI
Hosted context 64K tokens
Self-hosting About 64 GB of GPU memory in bfloat16; tested on one H200

Pro article

This story is for Pro members

You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.

Trending
  • No trending articles

Comments

avatar

Next Reads