NVIDIA's AIStore Stops GPU Clusters From Starving on Slow Data

AIStore (AIS) is NVIDIA's open-source, MIT-licensed distributed object storage system built from the ground up for AI workloads. It has been quietly maturing since 2017, and the recent v4.7 patch release caps a rapid series of major updates that have fundamentally reshaped how large-scale ML training pipelines can interact with storage. If your GPU cluster has ever sat idle waiting on data, this project is worth a serious look.

The Problem It's Solving

As AI and HPC datasets continue to increase in size, the time spent loading data begins to place a strain on total application performance. Fast GPUs are increasingly starved by slow I/O. The standard fix is to throw more workers at the DataLoader, but that only goes so far when the bottleneck is the storage layer itself. Traditional storage systems often cannot provide sufficient throughput for modern AI training workloads. Large language models may require 10+ GB/s sustained read performance, while computer vision training can demand even higher throughput for image and video data.

AIStore's answer is to treat storage as a first-class participant in the training pipeline, not just a dumb file server. AIS is a lightweight distributed storage stack tailored for AI applications. It's an elastic cluster that can grow and shrink at runtime and can be deployed anywhere from a single Linux machine to a bare-metal cluster of any size. Built from scratch, it provides linear scale-out, consistent performance, and a flexible deployment model.

What Makes It Different

AIS is a reliable storage cluster that can natively operate on both in-cluster and remote data, without treating either as a cache. It consistently shows balanced I/O distribution and linear scalability across an arbitrary number of clustered nodes. That last point matters: most distributed storage systems hit coordination bottlenecks as you add nodes. AIS is designed so that adding a node adds proportional throughput.

The feature set is dense. Here are the highlights most relevant to ML teams:

• Get-Batch API: A high-performance API for retrieving multiple objects and/or archived files in a single request. Behind the scenes, Get-Batch assembles the requested data from across the cluster and delivers the result as a continuous serialized stream. Regardless of retrieval source, Get-Batch always preserves the exact ordering of request entries in both streaming and buffered modes. This is huge for training pipelines that need deterministic batch ordering.