Skip to main content
Back

NYU's NeuroMesh Brings Decentralized Robot AI From Simulation to Real Hardware

A new open-source C++ stack unifies perception, control, and task assignment across mixed aerial-ground robot teams over decentralized mesh networks.

Paper, Full Access
  • NeuroMesh unifies multi-robot neural inference into a four-stage encode-message-aggregate-decode pipeline in C++ on ROS 2.
  • Dual reduction and broadcast aggregation lets one runtime cover both dense perception and compact coordination tasks.
  • Parallel three-stage pipeline decouples cycle time from end-to-end latency, bounded only by the slowest stage.
  • Zenoh peer mode delivered 4.8 ms latency and 0.3 percent packet loss over Doodlelabs mesh at 200 Hz.
  • Validated on heterogeneous aerial-ground teams for depth estimation, RL-based navigation, and Hungarian goal assignment.
  • Open-source release planned; paper and project site currently available.

Getting learned multi-robot policies out of simulation and onto real hardware is where most projects die. Every team ends up rewriting glue code for a single task, a single robot, and a single radio, and none of it transfers. A group from NYU, UC Berkeley, and the Army Research Laboratory is trying to fix that with NeuroMesh, a decentralized neural inference framework that runs the same execution stack whether the robots are estimating depth, avoiding collisions, or bidding on goals.

The accompanying paper lays out a four-stage abstraction, a dual-aggregation trick that lets one pipeline handle both dense perception and lightweight coordination, and a parallelized C++ runtime that reports single-digit millisecond latencies over a physical mesh network.

Why GNN papers rarely survive contact with hardware

Graph neural networks have become the default way to express multi-robot collaboration because they are naturally decentralized and match the topology of a robot team. Applications span collaborative perception, where robots exchange sensor information to improve scene understanding; cooperative control, where robots coordinate their actions; and planning and task assignment, where learned decentralized coordination improves resource allocation and mission execution. Most of these results live inside a single simulator or dataset.

A deployable system has to juggle perception, control, and decision-making at once, run on whatever compute a given robot happens to have, tolerate a flaky radio link, and stay flexible enough that swapping in a new model does not require rewriting the whole stack. Existing works typically focus on a single task family, platform, or communication setting, which leaves a gap between algorithmic demonstrations and reusable deployment infrastructure for heterogeneous teams.