LocalAI Shrinks Agents-A1 to Run Frontier Agentic Tasks on Consumer GPUs
APEX GGUF quantizations bring InternScience's 35B agentic MoE to consumer hardware with near-lossless quality at half the size of standard quants
Takeaways
- InternScience's Agents-A1 is a 35B MoE model claiming frontier-level agentic performance against GPT-5.5 and Kimi-K2.6.
- The APEX GGUF release by the LocalAI team brings it to consumer hardware via a novel MoE-aware quantization method.
- APEX assigns different precision levels per tensor type and layer, matching Q8 quality at roughly half the file size (~21 GB vs ~45 GB).
- Eight quantization tiers are available, from I-Balanced (best overall) to Mini (12 GB, fits consumer GPUs).
- The model supports 262K context, built-in vision, and native tool-calling; runs via llama.cpp, Ollama, LocalAI, or Docker with no code changes.
- Agents-A1 is trained in three stages: domain SFT, specialist teacher models per skill, then multi-teacher distillation back into one 35B student — free under Apache 2.0.
Running a frontier-class agentic model locally just got a lot more realistic. The Agents-A1-APEX-GGUF release packages InternScience's Agents-A1 , a 35B Mixture-of-Experts model purpose-built for agentic tasks , into a set of GGUF quantizations that can run on consumer hardware without sacrificing meaningful quality. The twist is the quantization method itself: APEX (Adaptive Precision for EXpert Models), a MoE-aware strategy from the LocalAI team that treats different parts of the model with different levels of compression, rather than squashing every weight to the same bit-width.
What is Agents-A1, and why does it matter?
Agents-A1 is a 35B Mixture-of-Experts agentic model from InternScience, built to scale heterogeneous agentic abilities across multiple domains including long-horizon search, engineering, scientific research, instruction following, and tool-calling. The MoE architecture means the model has 35B total parameters but only activates around 3B per token at inference time , keeping compute costs manageable while retaining a large knowledge base.
Agents-A1 took a different path from the trillion-parameter race: instead of scaling to trillion parameters, it scales agent horizon. The idea is that what makes an agent capable is not raw parameter count but the ability to sustain coherent, multi-step reasoning across long trajectories. The infrastructure handles agentic trajectories averaging 45K tokens with knowledge, action, observation, and verification traces.
Benchmarks that challenge models 30x its size
Despite operating in the ~35B model class, Agents-A1 delivers highly competitive performance against frontier-scale systems such as GPT-5.5, DeepSeek-V4-pro, and Kimi-K2.6. The numbers back this up across a wide spread of agent-oriented benchmarks:
- GAIA: 96.0 , a general-purpose agent benchmark requiring tool use and multi-step reasoning
- IFEval: 94.8 , instruction following under complex, multi-constraint prompts
- FrontierScience-Olympiad: 79.0 , graduate-level science problem solving
- BrowseComp: 75.5 , web browsing and information retrieval tasks
- HLE with tools: 47.6 , hard reasoning with external tool access
- SciCode: 44.3 , scientific coding tasks
These results show that Agents-A1 combines strong long-horizon search ability, robust scientific reasoning, and reliable instruction following, establishing it as a highly capable and efficient agentic model that narrows the gap with much larger frontier models. Early hands-on testers have been impressed: the tool calling is noticeably sharper than the original Qwen3.5-35B, and it follows instructions with surgical precision.