Exo Turns Four Mac Studios Into a 4.8TB/s AI Cluster

An open-source cluster tool just landed a spot on Apple's official product marketing pages, and the technical story behind it is worth unpacking. Exo, a tool for running frontier models across a mesh of Macs, is now featured on Apple's M5 Ultra Mac Studio and M6 / M5 Pro Mac Mini pages after a year of joint work on low-latency networking. The team reports that aggregate memory bandwidth across Macs now scales roughly linearly with RDMA, and a cluster of four M5 Ultra Mac Studios reaches around 4.8TB/s of aggregate memory bandwidth, territory that used to require a rack of data center GPUs.

The plumbing that makes this work

The unlock is a new macOS feature called RDMA, short for Remote Direct Memory Access. In plain terms, RDMA lets one machine reach into another machine's memory over the network without going through the CPU or the usual operating system stack. That collapses the round-trip cost of shuffling activations between nodes during inference. RDMA arrived in macOS 26.2, and it works on any Mac with Thunderbolt 5, including the M4 Pro Mac Mini, M4 Max Mac Studio, M4 Max MacBook Pro, and M3 Ultra Mac Studio.

The latency delta is the headline number. RDMA cuts latency between Macs from 300 microseconds to 3 microseconds, enabling fast tensor parallel inference across multiple machines. That two-orders-of-magnitude drop is what turns a stack of desktops from a science project into something that behaves like one machine with a shared pool of RAM.

Benchmarks: what actually changes

Real throughput numbers tell the story better than marketing math. On the same hardware, exo with RDMA hit 26.2 tokens per second across two nodes and 31.9 tokens per second across four, while the non-RDMA path managed only 17.2 and 15.2 tokens per second respectively. Without the new stack, going from two nodes to four made things slower, because network overhead grew faster than the added memory bandwidth helped. With RDMA, adding nodes finally scales the way it should. Apple's own framing is more conservative: a cluster of four Mac Studio systems delivers up to 3x faster AI inference than a single system.

Exo's tensor parallel implementation is what turns that plumbing into throughput. The project reports up to 1.8x speedup on two devices and 3.2x on four, using MLX as the inference backend and MLX distributed for communication between nodes.

Getting a cluster running

The setup ritual is short but weird. RDMA ships disabled and has to be flipped on per machine from Recovery mode:

1. Shut the Mac down and hold the power button for 10 seconds to reach the boot menu.
2. Choose Options to enter Recovery, then open Terminal from the Utilities menu.
3. Run rdma_ctl enable and reboot.
4. Wire every node to every other node with Thunderbolt 5 cables in a full mesh.
5. Launch exo, which handles automatic device discovery and model sharding.

A couple of gotchas worth knowing: every device in an RDMA cluster must be connected to every other, the cables must support TB5, and on a Mac Studio you cannot use the Thunderbolt 5 port next to the Ethernet port. Once running, exo exposes an OpenAI-compatible Chat Completions endpoint, a Claude Messages endpoint, and an Ollama-compatible API on http://localhost:52415, so existing clients keep working.

The models this puts within reach

The point of pooling memory across four Macs is running weights that no single machine can hold. Testing pushed models like Kimi K2 Thinking, which at over 600GB is too large for one Mac, plus DeepSeek V3.1 671B in 8-bit and Qwen3 235B in 8-bit, all as tensor-parallel workloads across four M3 Ultra Studios. DeepSeek V3, Llama 4 Maverick, and Qwen 3.6 are all Mixture-of-Experts models, where only a small fraction of parameters compute per token, so a cluster can hold the full weight set distributed across nodes and only ship the activations of the active experts between them. Thunderbolt-class bandwidth is enough because of that sparsity.

What it costs to build one

Exo's own math for a top-end M5 Ultra rig: four 256GB M5 Ultra machines wired with six Thunderbolt 5 cables (no special networking gear) run about $600 in cabling, for a total of 4 times $10,799 plus $600, or $43,796, yielding aggregate memory bandwidth of 4.8TB/s. Expensive, but the comparison point matters. The full four-Studio rig draws 450 to 600W total under load, with a Kimi K2 run coming in under 500W for the whole cluster, while a single NVIDIA H200 pulls about 700W by itself. An equivalent 1.5TB fast-memory NVIDIA build is measured in hundreds of thousands of dollars.

Where it still falls short

• Prefill is compute-bound and Apple Silicon has far less raw FLOPS than a data center GPU, so time-to-first-token on long prompts trails NVIDIA rigs.
• Linux support in exo currently runs on CPU only, so the interesting path is macOS-native.
• The RDMA setup requires macOS 26.2 or later and only works on Macs with Thunderbolt 5, ruling out older M-series machines.
• RDMA ports may fail to discover each other on mismatched macOS versions, so OS versions must match exactly (even beta version numbers) on all devices.

Exo has been chasing a disaggregated inference story too. By running compute-bound prefill on a DGX Spark, memory-bound decode on the M3 Ultra, and streaming the KV cache over 10GbE, they get the best of both hardware profiles with large speedups. Pair that with tensor-parallel decode on a Mac cluster and the shape of a plausible desk-side inference stack comes into focus, one where the bottleneck is no longer whether you can fit the model, but whether you can afford four Ultras.