Gadget Reinforcement Learning Slashes Quantum Circuit Errors by 10 Million Times
A new paper in Communications Physics teaches an RL agent to invent its own composite gates, cutting ground-state estimation errors by up to ten-million-fold on IBM Heron native gates.
ProTakeaways
- Kundu and Sarra introduce Gadget Reinforcement Learning in Communications Physics, blending RL with program synthesis.
- An agent invents composite gates (gadgets) on easy problems, then reuses them as actions on harder ones.
- Action space uses SX, X, RZ, and CZ, matching IBM Heron native gates and skipping NP-hard transpilation.
- Reports up to 10^7-fold error reduction on transverse-field Ising ground state estimation versus baseline RL.
- Scales to ten-qubit systems within roughly 2 to 3 days of GPU runtime.
- Code released at github.com/Aqasch/Gadget_RL, along with the arXiv preprint.
Designing quantum circuits by hand is a slog, and letting reinforcement learning do it usually breaks the moment you restrict the agent to real hardware. A new paper by Akash Kundu and Leopoldo Sarra, published in Communications Physics, proposes a fix: let the agent invent its own reusable gate sequences as it goes, then hand those back to itself as new actions. The authors call the method Gadget Reinforcement Learning (GRL), and they report error reductions of up to seven orders of magnitude on a canonical quantum physics benchmark.
Why circuit design keeps tripping up RL
Quantum computing promises useful simulation of complex quantum systems and optimization of large-scale combinatorial problems, but device noise and constrained qubit connectivity keep those promises in check. A parameterized quantum circuit (PQC) is a trainable circuit with tunable rotation angles, used as the ansatz inside variational quantum algorithms that estimate quantities like molecular ground states.
Different noise levels, connectivity topologies, and gate fidelities all shape what a good circuit looks like on a given chip, so any design method needs to be hardware-aware. Reinforcement learning fits the shape of the task, since building a circuit is a sequential decision problem. The catch is that existing RL approaches lose accuracy when restricted to hardware-native gates and device-level compilation. Force the agent to use only what the chip physically supports, and it struggles.