Sakana AI's Dream-Cubed Beats Real Minecraft Terrain 67% of the Time
Sakana AI and NYU release Dream-Cubed, a 30-billion-block Minecraft dataset and family of 3D diffusion models that generate infinitely large, fully playable worlds with block-level control.
- Dream-Cubed is a new dataset and model family from Sakana AI and NYU for generating playable Minecraft worlds at native block resolution.
- The dataset contains 30+ billion blocks from 1.8M+ procedurally generated chunks and 200K+ human-authored map chunks.
- Two diffusion models are trained: a discrete MD4 masked diffusion model and a continuous DDPM model, both on a ~280M parameter 3D DiT backbone.
- MD4's masking objective gives inpainting, outpainting, and block-conditioned generation for free with no extra fine-tuning.
- The best model was preferred over real Minecraft terrain 67% of the time in human evaluation studies.
- All code, pretrained models, and data are open-sourced on GitHub and HuggingFace.
Generative AI has conquered text, images, and video. But interactive 3D worlds -- the kind you can walk around in, build in, and break -- have remained stubbornly out of reach. Dream-Cubed, a new release from Sakana AI and NYU, takes a direct swing at this gap by treating Minecraft's building blocks exactly like language tokens: discrete, composable units that a large transformer can learn to predict.
The result is a family of diffusion models that can generate biome-accurate chunks of Minecraft terrain, inpaint missing regions, outpaint to arbitrary world sizes, and respond to hand-crafted block patterns as hard constraints -- all at the native block resolution, with outputs that are immediately loadable and playable in the actual game.
Why Minecraft is the right testbed
Minecraft is the best-selling video game in history, and its world representation is unusually clean for machine learning. Every location in the world is a voxel -- a single categorical value from a vocabulary of block types like stone, sand, or water. There are no continuous pixel values to worry about, no camera poses, no depth ambiguity. A 32x32x32 region of the world is just a tensor of integer block IDs.
This maps almost perfectly onto how language models work. Words are discrete tokens from a fixed vocabulary; blocks are discrete tokens from a fixed block vocabulary. The compositional structure that makes transformers so powerful for text turns out to be exactly the right inductive bias for 3D voxel worlds.
The dataset: 30 billion blocks and counting
Before training any models, the team had to solve a data problem. The paper describes a two-pronged collection strategy:
- Procedurally generated terrain: Over 1.8 million 32x32x32 chunks scraped from Minecraft's built-in world generator, labeled by biome (desert, ocean, jungle, village, and 11 others). The total comes to more than 30 billion individual blocks.