Southampton's MARLIN Teaches Robot Swarms to Negotiate Before They Learn
A hybrid framework lets robots debate their next move through LLM dialogue when their reinforcement learning policy is still too green to trust.
ProTakeaways
- MARLIN pairs MAPPO with LLM inter-robot negotiation to skip the unsafe early training phase in multi-robot RL.
- Each robot hosts its own Llama 3.1 8B Instruct instance and debates moves in natural language.
- A dynamic switcher picks between RL sampling and LLM planning based on recent performance and cached plans.
- At episode 100 MARLIN hit 1.0 versus 0.85 for MARL on the asymmetric two-slot corridor benchmark.
- Validated on TurtleBot3 hardware for the maze-like corridor, matching the simulation trend.
- Code available at github.com/SooratiLab/MARLIN, built on Ray RLlib.
Training a fleet of robots with reinforcement learning has a well-known ugly phase: the early episodes, when policies are essentially random and robots crash into walls, each other, or worse. Researchers from the University of Southampton have proposed a workaround called MARLIN (Multi-Agent Reinforcement Learning guided by Language-Based Inter-Robot Negotiation), which hands the wheel to a large language model until the RL policy is capable enough to take over.
The core idea is straightforward. Instead of letting a fresh MARL policy flail around for thousands of episodes, each robot is equipped with its own LLM instance. The robots talk to each other in natural language, propose moves, critique them, and agree on a plan. Those negotiated actions are then executed and fed back into the reinforcement learning trajectory as training data, so the policy still learns from them.
The cold-start problem in multi-robot RL
Multi-Agent PPO (MAPPO) has become the go-to algorithm for cooperative multi-robot systems because it introduces a centralised critic during training while maintaining decentralised execution, which helps handle the non-stationarity of multiple agents learning simultaneously. Training these policies for complex real-world scenarios often requires substantial data and time before reliable performance is achieved. During that window, robots behave unpredictably, which is fine in simulation but expensive or dangerous on real hardware.
Prior attempts to fuse LLMs with MARL have leaned on centralised architectures: an LLM generates reward functions, decomposes tasks, or masks action spaces from a single vantage point. MARLIN takes a different route by pushing the LLM into each individual agent and letting them negotiate peer-to-peer, more like a group chat than a dispatcher.