NYU's NeuroMesh Brings Decentralized Robot AI From Simulation to Real Hardware
A new open-source C++ stack unifies perception, control, and task assignment across mixed aerial-ground robot teams over decentralized mesh networks.
Takeaways
- NeuroMesh unifies multi-robot neural inference into a four-stage encode-message-aggregate-decode pipeline in C++ on ROS 2.
- Dual reduction and broadcast aggregation lets one runtime cover both dense perception and compact coordination tasks.
- Parallel three-stage pipeline decouples cycle time from end-to-end latency, bounded only by the slowest stage.
- Zenoh peer mode delivered 4.8 ms latency and 0.3 percent packet loss over Doodlelabs mesh at 200 Hz.
- Validated on heterogeneous aerial-ground teams for depth estimation, RL-based navigation, and Hungarian goal assignment.
- Open-source release planned; paper and project site currently available.
Getting learned multi-robot policies out of simulation and onto real hardware is where most projects die. Every team ends up rewriting glue code for a single task, a single robot, and a single radio, and none of it transfers. A group from NYU, UC Berkeley, and the Army Research Laboratory is trying to fix that with NeuroMesh, a decentralized neural inference framework that runs the same execution stack whether the robots are estimating depth, avoiding collisions, or bidding on goals.
The accompanying paper lays out a four-stage abstraction, a dual-aggregation trick that lets one pipeline handle both dense perception and lightweight coordination, and a parallelized C++ runtime that reports single-digit millisecond latencies over a physical mesh network.
Why GNN papers rarely survive contact with hardware
Graph neural networks have become the default way to express multi-robot collaboration because they are naturally decentralized and match the topology of a robot team. Applications span collaborative perception, where robots exchange sensor information to improve scene understanding; cooperative control, where robots coordinate their actions; and planning and task assignment, where learned decentralized coordination improves resource allocation and mission execution. Most of these results live inside a single simulator or dataset.
A deployable system has to juggle perception, control, and decision-making at once, run on whatever compute a given robot happens to have, tolerate a flaky radio link, and stay flexible enough that swapping in a new model does not require rewriting the whole stack. Existing works typically focus on a single task family, platform, or communication setting, which leaves a gap between algorithmic demonstrations and reusable deployment infrastructure for heterogeneous teams.