University of Exeter's AI Agent Masters Network Slicing Without a Single Reward Signal
A new framework lets language model agents master dynamic network control by reflecting on their own experience and self-finetuning, no reward engineering required.
- Researchers propose Self-Finetuning, a reward-free framework for LLM agents in continuous control, applied to 6G RAN slicing.
- An Actor-Reflector architecture replaces the RL Critic with a language model that writes trajectory-level reflections instead of scalar values.
- Refine-from-Reflection converts reflections into KTO preference datasets, distilling experience into model weights rather than prompts.
- Beats PPO, SAC, DQN and Reflexion on utility, cutting reconfiguration times by 59% versus PPO.
- Achieves this with one trajectory versus 1,600 for RL baselines, a massive sample-efficiency gain.
- Main limitation is LLM inference latency; authors suggest distilling into smaller deployable models for production.
Researchers at the University of Exeter and British Telecom have developed a language model agent for autonomous network control that learns from written critiques of its decisions. In a custom simulator, the system achieved the highest composite utility among five tested methods after one 300-step training trajectory. The reported reinforcement learning schedule used 1,600 trajectories, suggesting a substantial reduction in costly environment interactions.
The preprint applies the method to radio access network slicing, where a controller must divide spectrum among competing services in real time. “Reward-free” has a specific scope here: language judgments and preference optimization drive policy updates, while scalar metrics and a composite utility score remain part of the evaluation.
One Score for Conflicting Goals
Radio Access Network slicing allocates Physical Resource Blocks, the time-frequency units assigned by a cellular scheduler, across virtual network slices. The experiment includes guaranteed bit rate traffic, which has explicit service commitments, and elastic non-GBR traffic.
A controller must balance three objectives:
- Resource utilization: keep available spectrum productive.
- Service quality: limit violations across traffic classes.
- Stability: avoid frequent allocation changes and their operational overhead.
A conventional reinforcement learning reward compresses those objectives into one weighted scalar. Weight selection can produce policies that optimize one metric at the expense of another, and the useful balance may shift as traffic changes. Prompt-based agents can reason over written objectives, but long histories consume context and models may underuse information buried in earlier traces. Updating the model’s weights gives the agent a persistent form of adaptation.
A Critic That Writes
The framework separates online action selection from retrospective credit assignment. A Qwen3-4B Actor chooses resource allocations, while a DeepSeek-R1 Reflector reviews the completed trajectory. The loop proceeds through five stages:
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
