Skip to main content
Back

KAIST Study Shows Shared AI Coding Memory Boosts Performance by 3.7% Across Benchmarks

A new study shows sharing agent memories across unrelated coding benchmarks lifts average performance 3.7%, but only when memories stay abstract.

Paper, Full Access
  • KAIST, NYU, and DeepAuto.ai introduce Memory Transfer Learning, sharing coding-agent memories across heterogeneous benchmarks.
  • Cross-domain memory lifts average Pass@3 by 3.7% across six benchmarks with GPT-5-mini.
  • Abstract Insight memories transfer well; raw Trajectories cause negative transfer via brittle command reuse.
  • Gains come from meta-knowledge like validation routines, not from transferring algorithmic solutions.
  • Beats ReasoningBank and AgentKB while using just 431 memories versus AgentKB's 5,899.
  • Memories transfer across models (GPT-5-mini, DeepSeek V3.2, Qwen3-Coder); details at memorytransfer.github.io.

Most memory-augmented coding agents live in silos. An agent solving GitHub issues remembers only GitHub issues. An agent grinding through competitive programming only recalls competitive programming. A new study from researchers at KAIST, NYU, and DeepAuto.ai asks whether that separation is necessary, and reports that pooling memories across very different coding tasks yields a modest but consistent lift on six benchmarks, provided the memories are stored at the right level of abstraction.

The paper, Memory Transfer Learning, frames MTL as a systematic investigation rather than a new product. The headline number: cross-domain memory improves average performance by 3.7%, primarily by transferring meta-knowledge such as validation routines, rather than task-specific code. Code, prompts, and details are available on the project page.

Concept figure showing memory transfer learning across coding domains

The silo problem

Self-evolving agents work by saving something from each task they attempt, then retrieving those saved artifacts as hints on future tasks. In coding agents, memory is instantiated as code snippets, experiential knowledge including planning and debugging traces, or general programming principles. Leveraging this knowledge lets agents reference successful solution patterns on similar problems.

The catch is that existing systems typically only retrieve memories generated within the same benchmark. That ignores an obvious fact: despite the diversity of coding tasks, they share a common underlying infrastructure, including runtime environments such as Linux shells, programming languages, and cross-file dependency stacks. A trick learned on LiveCodeBench about writing quick inline verification tests should, in principle, help on SWE-Bench too.

Four flavors of memory

To test what actually transfers, the authors run the same coding agent (built on mini-swe-agent) across six benchmarks and generate memories in four formats spanning concrete to abstract: