Shanghai AI Lab's LifeSkill Lets Agents Rewrite Their Own Weights While Working

LifeSkill teaches LLM agents to bake experience into their weights at test time, beating all retrieval-based and RL baselines by 7 points on a lifelong learning benchmark.

·
·
·
Shanghai AI Lab's LifeSkill Lets Agents Rewrite Their Own Weights While WorkingPRO
  • LifeSkill is a two-stage RL framework that lets LLM agents update their own weights during deployment, internalizing skills from live task feedback instead of retrieving past experiences from memory.
  • It beats all retrieval-based and training-based baselines on LifelongAgentBench by 7 absolute points on average, reaching 0.82 accuracy on the Database environment.
  • Verifier-Guided Skill Learning rewards candidate skills by the average success rate of multiple rollouts that use them, creating an indirect but reliable training signal for skill quality.
  • Online Skill Internalization converts successful skill-conditioned trajectories into RL reward signals and updates the policy model's weights in real time, avoiding context bloat.
  • All experiments use Llama-3.1-8B-Instruct as the backbone; no public code repository has been released alongside the paper.
  • The approach scales with test-time compute: more rollouts per task push DB accuracy from 0.25 to 0.82, mirroring test-time scaling seen in reasoning models.

Most LLM agents today have a memory problem. They can retrieve past experiences from a database, but they never actually learn from them. Every new task starts from the same frozen weights, with the same frozen reasoning patterns. A new paper from East China Normal University and Shanghai AI Laboratory proposes LifeSkill, a framework that breaks this pattern by letting agents update their own parameters during deployment, turning interaction feedback into permanent knowledge.

The result is an agent that genuinely improves the longer it runs, without stuffing its context window with past trajectories and without requiring any offline retraining.

Why frozen agents keep failing

Lifelong learning is essential for intelligent agents operating in dynamic environments, yet current LLM-based agents remain stateless and unable to accumulate or transfer knowledge over time. The dominant workaround is experience retrieval: store past trajectories in a vector database and inject the most relevant ones into the prompt at inference time. It works, but it has a hard ceiling.

  • Retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows.
  • Conventional experience replay has limited effectiveness for LLM agents due to irrelevant information and context length constraints.
  • The agent treats every new task as a zero-shot problem, repeating the same errors regardless of how many prior attempts it has seen.

Existing lifelong learning agents for long-horizon tasks typically depend on discrete skill or past experiences retrieval with static parameters during inference, which prevents them from continuously internalizing test-time feedback like human learners. LifeSkill is designed to close exactly that gap.

The benchmark: three environments, one continuous stream

The paper evaluates on LifelongAgentBench, the first benchmark built specifically to measure whether agents can actually learn over time rather than just perform well on isolated tasks. It provides skill-grounded, interdependent tasks across three interactive environments -- Database, Operating System, and Knowledge Graph -- with automatic label verification, reproducibility, and modular extensibility.

Pro article

This story is for Pro members

You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.

Trending
  • No trending articles

Comments

avatar

Next Reads