UC San Diego's AIBuildAI Tops OpenAI's Kaggle Benchmark With 63.1% Medal Rate

A research team at UC San Diego has released AIBuildAI, an autonomous agent that takes a task description and a training dataset and hands back a trained model plus an inference script. It just claimed the top spot on MLE-Bench, OpenAI's benchmark of 75 real Kaggle competitions, with a medal rate high enough to place it above every prior autonomous ML engineering system on a suite curated to test real-world skills like training models, preparing datasets, and running experiments.

The headline number is 63.1%. When MLE-Bench launched, the best setup at the time was OpenAI's o1-preview with AIDE scaffolding, which earned a Kaggle bronze medal in 16.9% of competitions. AIBuildAI nearly quadruples that, and the authors argue it matches the capability of experienced human AI engineers.

Why AutoML kept hitting a ceiling

Building a competitive ML model is not one job. It is a sequence of very different jobs: sketching a modeling strategy, writing and debugging the training code, watching the loss curves, tweaking hyperparameters, and then ensembling whatever survives. Classic AutoML tools automated a slice of that pipeline, mostly hyperparameter search and model selection inside a predefined space, but they still required humans to define the search space and glue everything together.

The newer wave of LLM-based coding agents like AIDE, AIRA, and MLEvolve reframed the problem as a search over code. Each candidate solution is a script, and a single LLM call mutates one script into a slightly better one. This works, but it caps how much can change in any given step. AIBuildAI's authors argue that a single call cannot hold enough context to redesign an architecture, debug a training loop, and re-tune hyperparameters all at once. Their fix is to stop treating solution refinement as a one-shot code diff.