Zhejiang University's PILOT Boosts Small AI Math Accuracy by 8.9 Points
A hyper-network learns to inject a single steering vector into small LLMs, matching teacher guidance without the runtime cost.
Takeaways
- PILOT internalizes teacher-model strategic guidance as a single latent vector injected at one pivot layer.
- Achieves +8.9 points on MATH500 with Qwen2.5-1.5B and only 0.2% total inference latency overhead.
- A hyper-network synthesizes query-specific anchor vectors from a learnable Proto-Anchor prior warm-started with training centroids.
- Energy-Aligned Injection matches the anchor's norm to context tokens, preventing embedding shock and preserving representation geometry.
- Latent injection outperforms the same hints given as explicit text prompts, avoiding autoregressive parse-then-execute bottleneck.
- Code available at github.com/Chihaya-Anon-chan/PILOT; trains 7-39M parameters, roughly 0.5% of the backbone.
Small language models often know how to solve a problem but lose the plot halfway through. Give them a hint about the strategy from a larger model, and they suddenly succeed. Take away that hint at inference time, and they fall back into error propagation. A new method called PILOT tries to bake the strategic signal directly into the model's internal representations, using a lightweight side network rather than a full fine-tune.
The paper, PILOT: Planning via Internalized Latent Optimization Trajectories, comes out of Zhejiang University and Shanghai AI Lab, with code on GitHub. The headline number: on Qwen2.5-1.5B, PILOT lifts MATH500 accuracy by 8.9 points over zero-shot chain-of-thought, while adding roughly 0.2% to end-to-end inference latency.
The teacher-in-your-ear problem
Compact LLMs generally have the knowledge to solve multi-step problems, but they struggle with global planning. Without a global strategy, they generate myopically, and minor errors in early reasoning steps cascade into large deviations downstream, a phenomenon known as error propagation.
The usual fixes all have tradeoffs. Prompting a bigger model to hand over a strategy hint works, but adds real latency and a hard dependency on the teacher being available. Static parameter-efficient fine-tuning like LoRA bakes in fixed behavior that can't adjust per query. Activation steering methods rely on fixed vectors that ignore the query. Latent reasoning approaches like Coconut require invasive training that can disrupt the model's native representation space and cause catastrophic forgetting of pre-trained knowledge.
PILOT's authors want the accuracy of teacher guidance without any of those costs. Their bet: the strategic signal can be compressed into a single vector, synthesized on the fly from the query, and quietly injected into one layer of the frozen base model.