Skip to main content
Back

VRPCoder Beats Gemini-2.5-Pro at Logistics Code With Just 8B Parameters

VRPCoder, an 8B model trained with constraint-aware reinforcement learning, beats Gemini and Claude on vehicle routing code generation by catching bugs that objective-only testing misses

Paper, Full Access
  • Core idea: LLMs generating optimization solver code can silently omit constraints that don't affect the objective on test cases -- a bug standard testing misses.
  • Solution: "Constraint injection" uses feasible probes and one-constraint-violating probes to catch both over- and under-constraining bugs in generated code.
  • VRPCoder: An 8B model trained with this dual verifier achieves 93% Pass@1 on vehicle routing benchmarks, beating Gemini-2.5-Pro Preview on 3 of 4 benchmarks.
  • Margin: VRPCoder-GRPO exceeds Claude Sonnet 4.5 by 28 points and prior OR-focused LLMs by 78 points on average.
  • Training: The verifier is used both as a rejection-sampling filter during data synthesis and as a reward signal in GRPO reinforcement learning fine-tuning.
  • Benchmark: The paper introduces an expert-verified VRP benchmark suite covering 21 variants targeting Gurobi solver code generation.

When you ask an LLM to write solver code for a logistics optimization problem, how do you know it actually got the constraints right? The objective value might look correct on your test cases, but the model could have silently dropped a critical rule -- say, the one preventing trucks from exceeding their load capacity -- and you'd never know until a real delivery route fails. A new paper from researchers at Xizi Luo, Changhong He, and colleagues tackles this blind spot head-on, and the results are striking: their 8-billion-parameter model, VRPCoder, outperforms Gemini-2.5-Pro Preview on three out of four benchmarks and beats Claude Sonnet 4.5 by 28 percentage points on average.

The Invisible Bug Problem

The field of Operations Research (OR) -- the discipline of turning real-world logistics and planning problems into mathematical optimization models -- has been an exciting target for LLM automation. A specialized and highly impactful application of LLMs has emerged in OR, specifically using them for automated optimization modeling: translating natural language descriptions of real-world problems into precise mathematical formulations or directly into executable solver code.

The standard way to check whether that generated code is correct is differential testing: run the model's code and a reference solution on the same inputs, and see if the objective values match. This works well for simple problems. But for constraint-dense combinatorial problems like Vehicle Routing Problems (VRPs) -- where dozens of operational rules interact -- it breaks down badly.

Here's why: many constraints are non-binding on typical test instances. A constraint that says "no vehicle can visit more than 10 customers" might never be violated on a small 5-customer test case, even if the generated code forgot to include it entirely. The objective scores match, the test passes, and the bug ships. Systematic failure modes include constraint misunderstanding and a feasibility-optimality decoupling where models achieve high feasibility but zero optimality.