Skip to main content
Back

Free Random Projection Beats Standard RL Baselines by 40% Without Architecture Changes

A new input mapping built on free probability theory injects hierarchical bias into in-context RL agents without touching the model architecture.

  • New input mapping called Free Random Projection injects hierarchical bias into in-context RL via free-group word structure
  • Base orthogonal matrices are indexed by words so environments sharing a prefix share matrix factors
  • Beats standard random projection across five POPGym partial-observation tasks, with gains up to 40 percent on Stateless Cartpole
  • Theory shows effective kernel dimension strictly decreases with word length ℓ at fixed environment count
  • Linearly solvable MDP experiments confirm the largest gains occur on tree-structured state spaces
  • Code released at github.com/ThayaFluss/frp_rl, paper accepted to AISTATS 2026

Generalizing across environments remains one of the hardest problems in reinforcement learning. A new paper accepted to AISTATS 2026 proposes an elegant fix that lives entirely in the input pipeline: replace the standard random projection used to unify heterogeneous observation spaces with one whose randomness carries a built-in tree structure. The authors call it Free Random Projection (FRP), and it borrows machinery from free probability theory, an algebraic branch of random matrix theory that most machine learning researchers have never touched.

On multi-environment benchmarks, FRP consistently outperforms the standard random projection, and the mechanism is analyzed both through kernel spectra and through a solvable class of MDPs. No architectural change is required to plug it into an existing in-context RL agent.

The multi-environment glue problem

In-context reinforcement learning (ICRL) trains a single recurrent or transformer agent on many environments so that, at test time, it can adapt to a new task purely by processing observations, actions, and rewards inside its context window, without any gradient updates. Each environment often has its own observation and action dimensions. The standard trick, used in prior work like structured state-space ICRL agents, is to map every environment's raw inputs into a common vector space with a random orthogonal projection.

That approach treats each environment as an independent random rotation of the input, with no hierarchical or structural relationship between the projections used for different tasks. Yet many RL problems share tree-like latent structure: branching futures, nested subtasks, exponentially growing state possibilities. Prior work on hyperbolic RL has shown that explicit hierarchical geometry helps, but it typically requires custom latent spaces or architectures. FRP asks whether the same bias can instead emerge from the sampling distribution of the projection itself.

Words in a free group as an indexing scheme

The core idea is to stop drawing each environment's projection matrix independently and instead index projections by words in a free group