Google DeepMind has teamed up with Schmidt Sciences, the Cooperative AI Foundation, the UK government's Advanced Research and Invention Agency (ARIA), and Google.org to launch a $10 million research fund targeting one of AI's least-studied risks: what happens when millions of autonomous agents start talking to each other.

A Problem That Barely Exists Yet , But Will Soon

For years, AI safety research has focused on individual models. Can this model be jailbroken? Does it follow instructions? Is it aligned with human values? The emergence of advanced multi-agent systems presents risks that have been systematically underappreciated and understudied, in part because the deployment of such systems is currently rare, or constrained to highly controlled settings.

Google DeepMind, which made agent-based tools a centerpiece of Google I/O last month, has teamed up with several other organizations to announce a $10 million funding pot for researchers to study the behavior of multi-agent systems and come up with ways to prevent unsafe scenarios. Rohin Shah, who directs DeepMind's AGI safety and alignment research, says the window to get ahead of this is narrow. He estimates we have a few more months before agents are deployed throughout the economy in numbers that make potential risks a real concern.

Why Multi-Agent Safety Is Its Own Beast

The core insight driving this fund is that safety problems in multi-agent systems are not just harder versions of single-agent problems , they are categorically different. These risks are distinct from those posed by single agents or less advanced technologies, and will not necessarily be addressed by efforts to mitigate the latter.

Interactions between agents may introduce new failure modes or exacerbate existing vulnerabilities. Unlike single-agent architectures, where failures tend to remain localized, multi-agent systems exhibit complex interdependencies that can propagate or even amplify existing vulnerabilities. Think of it like traffic: you can make every individual car perfectly safe, and still end up with gridlock, pile-ups, or road rage at scale.