LocalAI Shrinks Qwen's AgentWorld-35B to Run on a Single 16 GB GPU
LocalAI's mudler releases APEX-quantized GGUFs of Qwen's agentic world model, enabling local inference on consumer hardware down to 12 GB VRAM.
Takeaways
- mudler (LocalAI team) released APEX-quantized GGUFs of Qwen's AgentWorld-35B-A3B, enabling local inference.
- APEX is a MoE-aware quantization method applying higher precision to edge layers and shared experts, lower to middle-layer routed experts.
- Eight quantization tiers available: from 23.6 GB (I-Balanced) down to 12.2 GB (I-Mini) for 16 GB VRAM GPUs.
- AgentWorld-35B-A3B is a language world model covering 7 agent domains (MCP, Search, Terminal, SWE, Android, Web, OS) with a 262K context window.
- APEX outperforms Unsloth Dynamic 2.0 on accuracy benchmarks while being 2x smaller; I-Quality matches Q8_0 at 21.3 GB vs 34.4 GB.
- Works out of the box with LocalAI, llama.cpp, Ollama, and LM Studio; Apache 2.0 license, free to use.
The LocalAI team's mudler has published APEX-quantized GGUF versions of Qwen's recently released AgentWorld-35B-A3B -- a 35B Mixture-of-Experts language world model -- making it runnable on local hardware ranging from a 24 GB workstation GPU all the way down to a 16 GB consumer card. The base model is a serious piece of work from Qwen; the announcement here is that mudler has done the compression work to make it accessible without a multi-GPU server.
What AgentWorld-35B-A3B actually is
Before getting into the quantization, it helps to understand what's being compressed. Qwen-AgentWorld-35B-A3B is a native language world model trained for agentic environment simulation -- the first to cover seven agent interaction domains within a single model. It simulates agentic environments via long chain-of-thought reasoning, predicting the next environment state given an agent's action and interaction history.
Think of it less as a chatbot and more as a simulator: you give it an agent's action and the current environment state, and it predicts what the environment looks like next. A single model covers MCP (tool calling), Search, Terminal, SWE (software engineering), Android, Web, and OS -- spanning both text and GUI interaction environments. It was trained through a three-stage pipeline: CPT injects environment knowledge, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity.
At the 35B-A3B scale, the three-stage pipeline lifts the overall average by +8.66 points on AgentWorldBench, bringing Qwen-AgentWorld-35B-A3B above Claude Sonnet 4.6. The model has a default context length of 262,144 tokens. In full precision (F16), that's a 64.6 GB file -- well beyond what most local setups can handle.