OptiLLM Boosts Any AI Model's Reasoning by 30 Points Without Fine-Tuning
An open-source proxy that stacks 20+ inference-time reasoning techniques onto any OpenAI-compatible endpoint, trading extra API calls for large accuracy gains.
PRO- OptiLLM is an OpenAI-compatible proxy adding 20+ inference-time reasoning techniques to any model, no fine-tuning needed.
- Activate techniques by prepending slugs like
moa-,mcts-, orplansearch-to the model name. - Reported gains include +30 points on AIME 2025 with MARS and GPT-4-level results from GPT-4o-mini on Arena-Hard-Auto.
- CePO with Qwen3 32B beats o3-mini and DeepSeek R1 on several math and code benchmarks.
- Tradeoff is 3-5x more tokens and latency per query, best suited for batch or critical queries.
- Works with OpenAI, Anthropic, Cerebras, Google, and 100+ models via LiteLLM.
Getting better reasoning out of an LLM usually means fine-tuning, swapping to a bigger model, or writing clever prompts. OptiLLM takes a different route: it sits in front of your existing API as a drop-in OpenAI-compatible proxy and spends extra tokens at inference time to squeeze more accuracy out of whatever model you already use. The project has crossed 4,000 stars and ships more than 20 techniques you activate by prepending a slug to the model name.
Instead of one API call, the proxy issues several with different reasoning strategies, then combines or picks the best answer. You pay in latency and tokens, and get accuracy back.
How the proxy actually works
You start the server, point your OpenAI client at http://localhost:8000/v1, and change the model string. Prefix moa- to activate Mixture of Agents, mcts- for Monte Carlo Tree Search, plansearch- for plan search, and so on. Everything else in your client code stays identical.
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1")
response = client.chat.completions.create(
model="moa-gpt-4o-mini",
messages=[{"role": "user", "content": "Solve: If 2x + 3 = 7, what is x?"}]
)Two other invocation styles work: pass optillm_approach inside extra_body, or wrap the slug in <optillm_approach>
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
