Baidu's ERNIE Evolve Rewrites Its Own AI Code and Boosts Speed 6x

Baidu has introduced ERNIE Evolve, an automated research system that edits code, runs experiments, and selects improvements for AI training and inference. The company describes it as its first concrete step toward recursive self-improvement, or RSI, in which AI helps improve its own software stack. The current system operates within human-defined repositories, objectives, evaluation rules, permissions, and compute budgets.

Each run starts with a code repository, an optimization target, and acceptance criteria. ERNIE Evolve modifies the code and tests candidates in isolated environments. A fixed evaluator scores each result, while a separate Reviewer component inspects the diff for invalid changes or benchmark exploits. Rejected attempts remain available as evidence for later hypotheses.

A two-loop search over code

ERNIE Evolve organizes its search into two nested loops. The outer loop selects research directions, and the inner loop develops and tests changes within each direction. Multiple lanes run in parallel, with each lane maintaining a tree of code versions so the system can extend promising branches and abandon weak ones.

Five components split the work

• Director: Selects a research direction and starting code version using an Experience Bank of previous runs and a Knowledge Base of papers and open-source projects.
• Selector: Chooses the next candidate to modify within a lane and records the hypothesis under test.
• Executor: Edits and runs the code inside an isolated environment.
• Evaluator: Applies a fixed scoring program stored outside the repository and hidden from the agents, reducing direct benchmark gaming.
• Reviewer: Examines the diff and scores for invalid modifications, misleading gains, or loopholes in the evaluation.

The evaluator rejects changes outside the permitted scope, and measured gains must exceed a configured noise threshold before a candidate can advance. These controls matter when small performance differences can result from run-to-run variance. They reduce common failure modes, although the reliability of the process still depends on the quality of the evaluator, tests, and scope restrictions.

Two benchmark wins and one miss

Baidu first tested the system on three public benchmark suites: NanoChat AutoResearch for model quality under a fixed training budget, NanoGPT Speedrun for training efficiency, and SOL-ExecBench for GPU kernel performance. According to Baidu’s results, ERNIE Evolve surpassed the previous public leader on the first two suites and fell short on SOL-ExecBench.

On NanoChat AutoResearch, ERNIE Evolve added two prediction branches beside the neural network. One branch uses n-gram statistics from the training corpus, while the other retrieves training documents that match the current context and uses their continuations as predictions. A learned gate combines both branches with the network output, allowing the model to use corpus patterns it has not absorbed during the benchmark’s five-minute training window.

NanoGPT Speedrun had already accumulated more than two years of community tuning and over 90 successive improvements. ERNIE Evolve reduced Track 1 training time to 38.9 seconds from the baseline of 39.9 seconds, with improvements to communication overlap, backpropagation, and output-layer gradient computation. On Track 3, it reached the target loss 100 steps earlier by combining weight decomposition with an optimization method that extrapolates from a moving average of previous parameter updates.

The SOL-ExecBench result provides a useful boundary on the claims. ERNIE Evolve produced a mean score of 0.819 against the public best of 0.827, showing that the automated search did not lead every task it attempted.

Production tests find larger gains

Baidu also applied ERNIE Evolve to the in-development ERNIE-5.5-Flash inference stack. These internal tasks began with systems already tuned by engineers, and proposed changes had to preserve correctness and pass production load tests. The results are company-reported and have not been independently reproduced.

Kunlunxin throughput rises 6.24 times

Across four research rounds on Baidu’s Kunlunxin P800 accelerator, ERNIE Evolve raised generation speed to 623.6% of the original baseline, equivalent to a 523.6% increase, while passing the stated correctness checks. Much of the gain came from enabling fused kernels for the model’s manifold-constrained hyper-connections, which combine operations to reduce memory traffic and launch overhead.

The system also enabled speculative decoding through the model’s multi-token prediction head. This technique drafts several tokens at once and verifies them together, reducing the number of sequential decoding steps. A separate fix corrected a dispatch bug that had routed decode requests through the wrong communication path.

Deterministic GPU inference recovers throughput

Baidu’s reinforcement-learning workload required each request to produce bitwise-identical tokens and log probabilities regardless of the other requests in its batch. SGLang’s deterministic mode preserves that property by fixing operation and reduction order, which typically lowers throughput.

• Throughput: 33.7% higher
• Time per output token: 26% lower
• Time to first token: 12% lower

The largest single improvement replaced deterministic sampling, which consumed 14% of GPU time per decoding step, with a Triton kernel that processes vocabulary chunks in parallel.

Small decoding changes add up

On a service already using speculative decoding, ERNIE Evolve improved single-request decoding speed by another 14.9%. Several useful modifications produced gains of only 0.3% to 0.5%, below the noise of individual measurements, so the system combined multiple candidates before testing whether the aggregate improvement was statistically credible.

One modification reformulated FP8 matrix multiplication by swapping operand roles, reducing the zero padding that dominated small-batch inputs. Another replaced a sequential scan in the mixture-of-experts scheduler with a two-level index, shortening the work required to map tokens to expert networks.

Why the production results matter

Automated machine-learning research is becoming a competitive field. Baidu compares ERNIE Evolve with systems from Recursive and Tencent’s Hyra-1.0, with each project using agents to propose, implement, and evaluate technical changes.

The production experiments show the practical value of bounded automated search. Kernel tuning, communication scheduling, numerical-format adjustments, and dispatch debugging often require many experiments for modest gains. A system that explores those changes in parallel can reduce engineering time, provided its tests reflect production behavior and every accepted diff receives appropriate review.

The evaluation design also offers a useful pattern for developers building similar systems: keep scoring logic outside the editable repository, restrict file and API access, measure experimental noise, record failed attempts, and separate candidate generation from review. Passing an automated evaluation establishes eligibility for adoption, while production deployment still requires operational testing and authorization.

Reproducibility has limits

Baidu has published the optimized benchmark artifacts on GitHub, allowing developers to inspect the resulting code and benchmark changes. The company has not released the ERNIE Evolve orchestrator, its complete prompts, the Experience Bank, or the internal production environments.

• Available: Selected optimized artifacts and reported benchmark results
• Documented: The high-level architecture, evaluation process, and several discovered optimizations
• Unavailable: The complete autonomous research system and internal production test setup

Reproducing the public results remains possible from the released artifacts, subject to hardware and environment differences. Evaluating the system’s search efficiency, failure rate, compute cost, and production claims requires access that Baidu has not provided.

RSI remains a bounded goal

ERNIE Evolve currently operates inside fixed boundaries supplied by people: a repository, an objective, an evaluator, permissions, and a resource budget. Its demonstrated capability is autonomous optimization within those constraints. Baidu treats expansion into training-system design, model capabilities, and evaluator generation as future work.

Baidu says later versions will target training infrastructure, then model behavior, and eventually the creation and refinement of evaluation suites. The company also calls for interruptibility, explicit limits on goals and resources, continuous safety evaluation, and human oversight. Its announcement leaves the technical enforcement mechanisms for those safeguards unspecified.