Apple's MLX Turns Four Macs Into a Trillion-Parameter Cluster
MLX's JACCL backend brings RDMA over Thunderbolt 5 to Mac clusters, delivering sub-50µs latency and up to 3x LLM inference speed across four Macs
PRO- Apple's MLX framework ships JACCL, a new distributed backend using RDMA over Thunderbolt 5 for Mac clusters.
- JACCL delivers sub-50µs latency and 50–60 Gbps throughput — an order of magnitude faster than the previous TCP-based ring backend.
- Four M3 Ultras running JACCL achieved 28+ tokens/sec on the 1-trillion-parameter Kimi 2.6 model and 3x fine-tuning throughput.
- Requires macOS 26.2 (developer preview available); RDMA must be enabled once per machine via macOS recovery mode.
- Supports tensor parallelism and pipeline parallelism; existing MLX code needs minimal changes to run distributed.
- JACCL is also usable as a standalone C++ library for RDMA collectives outside of ML workloads.
Running a trillion-parameter model on your desk used to be a punchline. With the JACCL backend now shipping in MLX and macOS 26.2, it is a real workflow. Apple's machine learning framework has quietly grown a distributed computing stack that turns a handful of Macs connected by Thunderbolt cables into a legitimate inference and fine-tuning cluster , no cloud required.
What just shipped
JACCL , the Jack and Angelos' Collective Communication Library , is a distributed backend that runs MLX collectives over RDMA on Thunderbolt 5. It was featured prominently at WWDC 2026 in the session "Explore distributed inference and training with MLX", and it is now documented and available in the open-source MLX repo. Starting from macOS 26.2, RDMA over Thunderbolt is available and enables low-latency communication between Macs with Thunderbolt 5.
The name JACCL (pronounced Jackal) stands for Jack and Angelos' Collective Communication Library , an obvious pun on Nvidia's NCCL, but also a tribute to Jack Beasley who led the development of RDMA over Thunderbolt at Apple. The positioning is deliberate: this is Apple's answer to the GPU cluster networking problem, running the same all-reduce, all-gather, and broadcast primitives that power datacenter training , just over Thunderbolt 5 instead of NVLink or InfiniBand.
Why RDMA changes everything here
The previous way to cluster Macs with MLX used a ring-allreduce backend over TCP sockets. It worked, but TCP introduces CPU and OS overhead on every transfer. Starting in macOS 26.2, RDMA (Remote Direct Memory Access) is supported over Thunderbolt 5. RDMA moves data directly from one machine's memory to another's, avoiding most CPU and operating system overhead.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
