LocalAI Shrinks Nex-N2-mini to Run on a 12GB Consumer GPU
The LocalAI team's APEX quantization brings a 35B agentic coding MoE with vision and 262k context to consumer GPUs starting at 12 GB VRAM.
PRO- The LocalAI team released APEX GGUF quantizations of Nex-N2-mini, a 35B agentic MoE model that runs on consumer GPUs starting at 12 GB VRAM.
- APEX is a MoE-aware quantization method that compresses middle layers aggressively while keeping edge layers sharp, beating Q8_0 quality at roughly half the file size.
- Nex-N2-mini scores 50.2 on SWE-Bench Pro and 60.7 on Terminal-Bench 2.1, with built-in tool calling, function invocation, and multi-step reasoning.
- The model includes a 262k-token context window and a bundled vision encoder (mmproj.gguf) for image understanding.
- Eight quantization tiers are available; I-Balanced is recommended for general use, I-Compact for tight VRAM budgets.
- Free under Apache 2.0; run with a single command via LocalAI, llama.cpp, Ollama, LM Studio, or Docker.
Running a capable agentic coding model locally has always meant a painful hardware tax. Nex-N2-mini, the open-source 35B-parameter model from Nex AGI, is purpose-built for autonomous coding, tool use, and long multi-step workflows. The catch: in its original form it needs serious server-grade hardware. Now the LocalAI team has released APEX GGUF quantizations that change that equation entirely.
A model built for agents, not chat
Nex-N2-mini is a lightweight open-source agentic model built on Qwen3.5-35B-A3B-Base, with 35B total parameters but only 3B active per token. It shares the Agentic Thinking framework of the larger Nex-N2-Pro at lower latency and resource cost for coding and agent workflows.
Rather than treating reasoning, tool use, and environment execution as separate capabilities, Nex-N2 unifies them through an Agentic Thinking framework that connects requirement understanding, task planning, code implementation, environmental feedback, evaluation and debugging, and continuous iteration into a single closed loop. In practice, this means the model can write code, run it, read the error, fix the bug, and keep going without you holding its hand.
The framework has two key mechanisms:
- Adaptive Thinking: the model decides on its own when to think and how deeply, executing simple actions quickly while reasoning thoroughly on critical decisions.
- Coherent Thinking: one consistent reasoning paradigm across general reasoning and diverse agentic tasks, staying consistent across tasks and modalities to enable stable capability transfer.
What the benchmarks actually say
Nex-N2-mini scores 50.2 on SWE-Bench Pro and 60.7 on Terminal-Bench 2.1, making it suitable for automated bug fixing, code review, terminal command execution, and iterative debugging workflows where the model must execute commands, parse errors, and adapt its approach based on feedback. Scoring 74.1 on BrowseComp and 62.0 on WideSearch, the model also excels when applications require function calling, web search integration, or sequential tool use.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.