LocalAI Shrinks Ornith-1.0-35B to Run on a Single Consumer GPU
Ornith-1.0-35B, a self-scaffolding agentic coding MoE, gets APEX quantization for consumer GPU deployment with near-lossless compression
Takeaways
- Ornith-1.0-35B-APEX-GGUF is a quantized, locally-runnable build of DeepReinforce AI's agentic coding model, released under MIT license.
- Ornith-1.0-35B scores 75.6 on SWE-Bench Verified and 64.2 on Terminal-Bench 2.1, beating models up to 10x its size.
- The model uses self-scaffolding RL: it learns to design its own task plans and tool-call workflows, not just solutions.
- APEX quantization applies different precision levels per tensor type and layer position, achieving Q8_0 quality at 38% smaller size.
- Eight GGUF variants are available for different hardware budgets, from consumer GPUs to the I-Quality tier at ~21 GB.
- Runs via llama.cpp, Ollama, LM Studio, or LocalAI; compatible with OpenAI-style tool-calling APIs.
A powerful agentic coding model just got a lot more accessible. The LocalAI team has released Ornith-1.0-35B-APEX-GGUF, a suite of quantized builds of DeepReinforce AI's Ornith-1.0-35B that can run on a single consumer GPU. The story here is actually two stories in one: a genuinely impressive open-source agentic coding model, and a smarter-than-usual quantization method purpose-built for the architecture it runs on.
The model underneath
Ornith 1.0 is a family of open-source large language models built by DeepReinforce AI specifically for agentic coding, spanning four parameter sizes -- 9B Dense, 31B Dense, 35B MoE, and 397B MoE -- all under MIT license with no regional restrictions. The 35B variant is the mid-tier option: big enough to be competitive, small enough to run locally.
Ornith-1.0-35B is a 35B Mixture-of-Experts reasoning model post-trained on Qwen 3.5 using a self-improving RL framework that jointly learns solution rollouts and the task-specific scaffolds guiding them. MoE (Mixture-of-Experts) means the model has many specialized sub-networks ("experts"), but only a small subset activates for any given token -- keeping compute low despite the large parameter count.
Self-scaffolding: the real innovation
The core innovation behind Ornith 1.0 is self-scaffolding reinforcement learning. Traditional coding agents rely on human-designed harnesses -- fixed workflows for tool calls, error recovery, and task decomposition. Ornith 1.0 treats the scaffold as a learnable object that co-evolves with the model's policy during RL training, meaning it generates its own task plans, launches tools, inspects intermediate results, and rewrites failing steps without human intervention.
The core innovation is a self-improving RL training framework: rather than relying on fixed human-designed harnesses, the model co-learns solution rollouts and the task-specific scaffolds that guide them. Reward is propagated to both stages, enabling per-task orchestration strategies to emerge automatically. In plain terms: the model learns not just how to solve a coding task, but