Tinygrad Beats AMD's Own Stack on MI350 Training Llama 3.1 8B

The tiny corp just posted its first state of the art result on MLPerf Training's Llama 3.1 8B benchmark, clocking 2 hours 6 minutes and edging out the 2h 7m time from AMD's own optimized Docker submission running on the same physical machine. The margin is razor thin, but the story is not. A tiny, pure Python neural network library beat the vendor's hand tuned stack on the vendor's own hardware.

The team also noted the thermal context around the run. It is warm in San Diego and, as they put it in the announcement thread, they are too cheap to buy AC. Their telemetry chart shows a visible thermal throttle, which means wall clock time is still leaving performance on the floor. The next target they called out is NVIDIA's 82.2 minute run on 8x B200, roughly 35 percent faster than the current tinygrad number.

What actually got benchmarked

MLPerf Training v5.1 introduced Llama 3.1 8B as a new pretraining benchmark, replacing BERT with a modern model that can still run on single-node systems. This matters because the Llama 3.1 405B benchmark added in MLPerf Training v5.0 required a minimum of 256 GPUs per submission, creating barriers for organizations looking to benchmark their systems without massive GPU clusters. The 8B benchmark brings modern LLM pretraining back within reach of a single node.

The reference workload is pretraining on the C4 English dataset with a specified convergence target. A compliant time-to-train score requires 10 consecutive runs, timings are calculated using run_start and run_stop timestamps, and the slowest and fastest runs are dropped before averaging the remaining eight. That makes single number comparisons like 2h 6m vs 2h 7m meaningful in a way that microbenchmarks are not.

The real breakthrough is in the driver

Tinygrad has been chipping at the AMD software stack for a while, and the recent MLPerf submissions leaned on a piece of infrastructure most frameworks do not have. According to MLCommons' supplemental discussion, tiny corp submitted LLaMa 8B pretraining results on single-node AMD MI350 machines, leveraging tinygrad with its python-based userspace driver for the MI350, which delivered a performance uplift over the kernel driver through reduced overhead and more control over the hardware.

That driver, called AM, is doing something unusual. AM is a userspace driver targeting AMD's RDNA3/RDNA4 hardware where you only need tinygrad to send compute tasks to your GPU, with the amdgpu kernel module unloaded and DEV=AMD set. There is also a pure Python driver variant that talks directly to /dev/kfd and /dev/dri/renderD* via ctypes ioctls, bypassing the ROCm/HIP userspace stack, with a pluggable architecture for future bare-metal PCI backends. In practical terms, tinygrad is skipping ROCm entirely, which is where a lot of framework overhead and instability tends to live on AMD.

Why this result is a big deal

Beating AMD's own submission on AMD's own hardware is a specific kind of statement. Here is what it demonstrates:

• Framework overhead is not inevitable. A Python first stack can match or beat vendor C++ pipelines when kernel scheduling and driver overhead are controlled.
• ROCm is not the only path to MI350 performance. A userspace driver going straight to KFD ioctls is viable at MLPerf scale.
• The gap to NVIDIA is measurable, not mythical. The B200 target of 82.2 minutes vs 126 minutes on MI350 gives a concrete number to close.

Where tinygrad shines and where it does not

Tinygrad's design philosophy has always been minimalism. The core features are lazy tensors for aggressive operation fusion, custom kernels for shape specialization, and a simple backend that is easy to optimize. The tradeoffs are real. Tinygrad is a fit for learning deep learning fundamentals, quick prototyping, edge deployment on limited hardware, and running LLMs on consumer hardware, while PyTorch remains the better choice for production-scale training, research with complex architectures, and maximum compatibility.

The framework runs on NVIDIA, AMD, Apple Metal, and Qualcomm backends, with MLPerf implementations tested via GitHub Actions workflows. It also powers openpilot's driving model on the Snapdragon 845 GPU, replacing SNPE with a faster stack that supports loading ONNX files, supports training, and allows for attention.

Practical takeaways

If you are running AMD GPUs and have been frustrated by ROCm, tinygrad is now a serious option for training and not just inference. The framework is MIT licensed and the code sits in the tinygrad repository. The MLPerf implementations live under examples/mlperf/, so you can look at the exact training loop that produced the 2h 6m number.

For teams evaluating hardware, the practical read is that AMD MI350 hardware is genuinely competitive on modern LLM pretraining when the software stack cooperates, but NVIDIA B200 still holds a meaningful lead on wall clock. And if you are running compute in a hot room without AC, the thermal ceiling matters as much as the software.