TU Delft's LLM-TALE Teaches Robot Arms to Self-Correct With 93.3% Success
A new framework combines LLM task planning with residual reinforcement learning, hitting 93.3% real-world success where pure LLM control gets 0%.
ProTakeaways
- LLM-TALE combines LLM task planning with residual RL, using critic values to pick among multiple affordance modes proposed by the model.
- Real-robot PutBox task: 93.3% success zero-shot from sim, versus 0% for an LLM-only controller that collides with the cupboard.
- Beats RLPD (with 25 demos) and Text2Reward on sample efficiency across six ManiSkill and RLBench tasks with both TD3 and PPO.
- Three-prompt pipeline (task planner, affordance identifier, affordance planner) using GPT-4o is queried once and cached as Python functions.
- PD controller acts as base policy, RL policy learns residual corrections; the paper shows behavior-cloned base policies collapse under exploration.
- Code and prompts available at github.com/llm-tale/llm_tale; accepted to ICRA 2026.
Getting a robot arm to learn manipulation tasks from scratch is painful. Reinforcement learning agents need enormous amounts of interaction to figure out that yes, you have to grasp the cube before you can move it. A new paper from TU Delft and RWTH Aachen, LLM-TALE, tries to fix this by letting a language model sketch out the plan while an RL policy handles the physics the LLM cannot reason about.
The framework trains faster than strong demonstration-based baselines, and transfers zero-shot from simulation to a real Franka Panda arm with a 93.3% success rate on a tricky pick-and-place task where a naive LLM controller fails every single time.
Why LLMs alone keep dropping the box
Two approaches have dominated the recent literature on using foundation models for robotic RL. The first uses language models to write dense reward functions, like Text2Reward or Eureka. The second proposes actions directly and dumps them into an off-policy replay buffer. Both share a weakness: their understanding of the physical world is incomplete and can yield erroneous guidance, while hand-crafted reward signals can produce reward hacking behaviors where the robot sits in a high-reward region without ever finishing the task.
The authors use a running example they call PutBox: place an open box inside a cupboard. An LLM will happily suggest grabbing the container from the top and placing it vertically. Both moves are semantically fine. Both are physically catastrophic. Top-grasping blocks the placement motion, and vertical placement either spills the contents or wedges the box out of reach. The model has no way to know this without actually trying.