Perplexity Turns AI Failures Into Training Data, Cutting Tool Errors 21%

Perplexity has published a post-training recipe that learns from both successful and failed sessions generated by its Computer agent. The method, detailed in a research post, combines rejection sampling fine-tuning with hint-guided self-distillation. In a live A/B test between two checkpoints trained with the combined approach, the company recorded a statistically significant 21.2% relative reduction in tool-call failures.

Failed traces become training targets

Rejection sampling fine-tuning, or RFT, grades production sessions, retains successful examples, and trains the model to imitate their trajectories. Successful sessions can still contain weak intermediate steps, while discarded failures often reveal schema violations, incorrect arguments, and missed user constraints.

Perplexity assigns each turn to one of three training paths:

• Imitate: Non-error turns from successful sessions receive cross-entropy loss, which increases the probability of the recorded tokens.
• Correct: Error turns with validated hints receive Kullback-Leibler divergence loss, which moves the model’s token distribution toward its own hint-conditioned predictions.
• Preserve as context: Turns from unsuccessful sessions without validated hints remain in the input history but contribute no training loss.

This routing preserves useful context without teaching the model to reproduce every action in a failed trajectory.

One checkpoint plays both roles

Hint-guided self-distillation adapts the method described in the OPSD paper. The same GLM 5.2 checkpoint serves as teacher and student through two forward passes over the recorded session.

1. The teacher pass receives the corrective hint.
2. The student pass receives the original context without the hint.
3. Both passes use teacher forcing, meaning they predict tokens along the recorded trajectory.
4. The teacher predictions are detached from gradient computation and used as fixed targets.
5. The student learns to reproduce the teacher’s next-token distribution from the original context alone.

At inference time, the model receives no hint. Training is intended to internalize the correction so the original context becomes sufficient to produce the better action.

Hints are limited to prior evidence

Each hint is a short corrective instruction supported by information available before the mistaken turn. The pipeline checks that the hint’s claims follow from the earlier context before admitting it into training.

This temporal constraint reduces hindsight bias. For example, a preference disclosed for the first time in a user complaint cannot be converted into an instruction that the model supposedly should have followed earlier.

Errors require careful attribution

Perplexity draws sessions from a training-eligible pool served by GLM 5.2, with opt-outs and sessions flagged for personally identifiable information excluded upstream. An LLM judge rates task difficulty before examples are divided into two sources:

• User feedback: Sessions containing follow-up corrections or dissatisfaction signals.
• Tool errors: Sessions containing failed calls identified by rules or model-based review.

Two additional LLM judges assess whether the final response satisfied the user’s request. For complaint-driven examples, three judges identify the turn or turns responsible for the failure, and at least two must agree. Perplexity reports that the final assistant turn before a complaint caused the problem only about half the time.

Known tool violations use deterministic checks. One example is passing recency_filter: "year" when the tool schema accepts only day, week, or month. Ambiguous failures are sent to an LLM judge.

Hints pass a preflight test

Before training, the team tested whether hints improved the base model’s behavior on held-out examples. Across 985 tool-error turns, adding hints increased failure avoidance from 75.1% to 93.7%, a gain of 18.6 percentage points. The share of turns taking the corrected action rose from 60.6% to 82.3%.

On user-feedback turns, the proportion judged fixed or moving toward a fix roughly doubled when the model received a hint. These measurements test whether the hints contain useful corrective signal; they do not measure what the model retains after distillation.

Tool failures fall in production

The offline checkpoints were trained on different data, so their results cannot isolate the contribution of OPSD. Task-level benchmark scores were also mixed, indicating that lower tool-error rates did not consistently produce higher overall task success.

The live A/B test assigned roughly 100,000 users to each checkpoint. Among tool calls with a recorded status, failures declined by 0.47 percentage points, from 2.24% to 1.77%. That equals the reported 21.2% relative reduction and reached statistical significance.

Strong dissatisfaction averaged 2.58% for the first checkpoint and 2.54% for the second. The difference was not statistically significant, leaving the production evidence stronger for tool reliability than for user satisfaction.

A practical recipe for agent teams

• Record structured failures: Tool statuses, schema violations, invalid arguments, and user corrections provide specific supervision.
• Attribute errors to turns: Training against the final response alone can target the wrong action.
• Enforce temporal grounding: Corrective hints should use only information available when the error occurred.
• Mask uncertain examples: Unvalidated turns can remain as context without contributing loss.
• Retain an imitation anchor: Cross-entropy training on successful behavior helps prevent teacher and student from matching by ignoring relevant context.

The approach converts production failures into distillation targets using the same underlying checkpoint, without requiring handwritten reference answers or a separate, stronger teacher. Its current limitations remain substantial: automated judges can misclassify errors, a corrected turn may still lead to a failed task, and the reported satisfaction metric showed no significant gain. The clearest result is narrower and measurable: grounded corrections reduced recorded tool-call failures in live traffic.