---
title: "LocalAI Shrinks Qwen's AgentWorld-35B to Run on a Single 16 GB GPU"
subtitle: "LocalAI's mudler releases APEX-quantized GGUFs of Qwen's agentic world model, enabling local inference on consumer hardware down to 12 GB VRAM."
url: "https://alphasignal.ai/news/localai-shrinks-qwen-s-agentworld-35b-to-run-on-a-single-16-gb-gpu"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-07-02T12:53:52.000Z"
updated: "2026-07-09T02:49:05.096Z"
type: "Model"
topics:
  - "Llms"
  - "Gpus"
subtopics:
  - "Mixture Of Experts"
  - "Vision Language"
  - "Quantization"
upvotes: 2567
source_url: "https://huggingface.co/mudler/Qwen-AgentWorld-35B-A3B-APEX-GGUF"
comments: 0
truncated: true
---

# LocalAI Shrinks Qwen's AgentWorld-35B to Run on a Single 16 GB GPU

> LocalAI's mudler releases APEX-quantized GGUFs of Qwen's agentic world model, enabling local inference on consumer hardware down to 12 GB VRAM.

The [LocalAI](https://github.com/mudler/LocalAI) team's mudler has published [APEX-quantized GGUF versions](https://huggingface.co/mudler/Qwen-AgentWorld-35B-A3B-APEX-GGUF) of Qwen's recently released AgentWorld-35B-A3B -- a 35B Mixture-of-Experts language world model -- making it runnable on local hardware ranging from a 24 GB workstation GPU all the way down to a 16 GB consumer card. The base model is a serious piece of work from Qwen; the announcement here is that mudler has done the compression work to make it accessible without a multi-GPU server.

## What AgentWorld-35B-A3B actually is

Before getting into the quantization, it helps to understand what's being compressed. Qwen-AgentWorld-35B-A3B is a native language world model trained for agentic environment simulation -- the first to cover seven agent interaction domains within a single model. It simulates agentic environments via long chain-of-thought reasoning, predicting the next environment state given an agent's action and interaction history.

Think of it less as a chatbot and more as a simulator: you give it an agent's action and the current environment state, and it predicts what the environment looks like next. A single model covers MCP (tool calling), Search, Terminal, SWE (software engineering), Android, Web, and OS -- spanning both text and GUI interaction environments. It was trained through a three-stage pipeline: CPT injects environment knowledge, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity.

At the 35B-A3B scale, the three-stage pipeline lifts the overall average by +8.66 points on AgentWorldBench, bringing Qwen-AgentWorld-35B-A3B above Claude Sonnet 4.6\. The model has a default context length of 262,144 tokens. In full precision (F16), that's a 64.6 GB file -- well beyond what most local setups can handle.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [mudler/Qwen-AgentWorld-35B-A3B-APEX-GGUF · Hugging Face](https://huggingface.co/mudler/Qwen-AgentWorld-35B-A3B-APEX-GGUF)
- [APEX — Adaptive Precision for EXpert Models (GitHub)](https://github.com/localai-org/apex-quant)
- [Qwen/Qwen-AgentWorld-35B-A3B · Hugging Face](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B)
- [mudler/Qwen3.5-35B-A3B-APEX-GGUF · Hugging Face (benchmark reference)](https://huggingface.co/mudler/Qwen3.5-35B-A3B-APEX-GGUF)
- [Qwen-AgentWorld Blog Post](https://qwen.ai/blog?id=qwen-agentworld)