RAT Automates Code Environment Setup Across 5 Languages, Beating Rivals by 36%
RAT is a multi-language agent framework that automatically configures executable environments for any code repository, beating prior baselines by 36.1% on environment setup success rate.
Takeaways
- RAT (RunAnyThing) is a new agent framework that fully automates executable environment configuration for arbitrary code repositories across multiple programming languages.
- It improves Environment Setup Success Rate (ESSR) by 36.1% on average over strong baselines, including Python-only systems like Repo2Run.
- RAT uses a four-stage pipeline: language-aware abstraction, base image initialization, a specialized toolset, and an isolated sandbox with iterative repair.
- The authors also release RATBench, a new benchmark covering Python, Java, JS/TS, Rust, and Go repositories of varying sizes for rigorous evaluation.
- Prior systems like Repo2Run were restricted to Python; RAT is the first to tackle fully automated multi-language environment setup without pre-existing Dockerfiles or CI artifacts.
- The paper is available on arXiv; no public code release has been announced yet.
Getting an AI coding agent to actually run a repository it has never seen before is harder than it sounds. Before any agent can fix a bug, write a test, or refactor code, it needs a working environment: the right language runtime, the right package versions, the right build toolchain. For most repos, that setup is still done by hand. A new paper from researchers Renhong Huang, Dongdong Hua, Yifei Sun, Sitao Ding, Hanyang Yuan, Daixin Wang, and Yang Yang proposes to change that with RAT (RunAnyThing), a fully automated agent framework that can configure executable environments across arbitrary repositories and programming languages.
The wall every code agent hits
The modern wave of autonomous software engineering agents -- SWE-agent, Devin, and their relatives -- all share a dirty secret: they depend on pre-configured environments. Benchmarks like SWE-bench sidestep the problem by shipping hand-crafted Dockerfiles alongside each task. Repository-level efforts better reflect real-world practice, but they ignore the environment configuration task by providing manually configured Docker files. That is a significant gap. In the wild, a new repository arrives with no such gift.
A handful of systems have tried to close this gap. Repo2Run is an LLM-based agent aiming at automating the building of executable test environments for any repositories at scale. It was evaluated on a benchmark of 420 Python repositories with unit tests, achieving an 86.0% success rate and outperforming SWE-agent by 77.0%. But Repo2Run is Python-only. ExecutionAgent generates scripts for building and testing projects, while RepoLaunch and SetUpAgent employ bash-interactive agents. Each of these approaches either locks into a single language or leans on pre-existing artifacts like CI logs or known Dockerfiles. RAT is designed to need none of that.