Google Research Finally Explains Why Diffusion Models Don't Just Copy Training Data

Diffusion models are among the most powerful tools for generative tasks requiring complex structure, like image generation and molecular discovery, and they have shown an exciting ability to generalize beyond their training data. But nobody could fully explain why. A new paper from Google Research, presented at ICLR 2026, now offers a rigorous mathematical answer.

The core finding is that a model's creativity is not a random fluke. Instead, it is a consequence of how neural network training naturally "smooths" the transformation from noise back to data during the generation process. In short: the very imperfection of how neural networks learn is what makes diffusion models creative.

The memorization trap

Training a diffusion model begins with taking real data samples and intentionally corrupting them with noise until they become unrecognizable. The model is then trained to reverse this corruption step-by-step, a process called denoising. If the model learns to do this perfectly based only on its training samples, it should produce carbon copies of them during deployment , a behavior known as memorization.

The scientific community has been divided on whether these models are truly creative or merely imitate what they have seen. An even more pressing concern is memorization, where the model's output partially or fully resembles training data, raising serious implications for data privacy. The memorization phenomenon has been observed in practice when models have large capacities relative to the training set size, which likely results in too good an approximation to the empirical score function.

So what stops a perfectly-trained model from just being a very expensive copy machine? The answer turns out to live in the mathematics of the score function , the learned "force field" that guides noisy data back toward meaningful outputs during generation.

Score smoothing: the accidental engine of creativity

The score function (SF) is the gradient of the log-probability of the data at any given noise level. Think of it as a vector field where every point in noisy space has an arrow pointing toward the nearest plausible image. At the core of diffusion models is the training of neural networks to fit a series of target functions called the empirical score functions (ESFs), which drive the denoising process at inference time. When equipped with the exact ESF instead of the version learned by neural networks, the diffusion model ends up generating data points that already exist in the training set , memorization.

The key discovery is that imperfect training due to regularization naturally leads to a slight blurring of the learned score function in a process called "score smoothing." Regularization here refers to standard training techniques like weight decay (also known as L2 regularization), which penalizes large parameter values and prevents the network from fitting the training data too precisely. The result is a score function that is slightly smoother , with softer transitions instead of sharp cliffs , and this smoothness is exactly what drives the model to generate novel outputs.

A 1D thought experiment that makes it concrete

The paper uses a beautifully simple example to illustrate the mechanism. Imagine a 1D world with only two training points: +1 and -1. The perfect score function creates a sharp boundary at zero: everything to the left gets pulled to -1, everything to the right gets pulled to +1. Every generated sample collapses onto a training point. Memorization.

A diffusion model's creativity is a mathematical consequence of neural networks learning a "smoothed" version of the score function, driving the model to interpolate between training data points along the hidden data manifold. In the 1D example, the smoothed score function softens that sharp boundary into a gentler slope. Particles that start near the center slow down and settle between the two training points , generating genuinely new data.

The team trained two-layer ReLU networks with the AdamW optimizer under varying levels of weight decay to validate this. The stronger the weight decay, the smoother the learned score function, and the larger the "interpolation zone" between training points. Crucially, they also showed that neural network learning is just one way to achieve score smoothing, motivating the exploration of alternative score estimators that facilitate generalization in diffusion models. Even without explicit regularization, implicit regularization from gradient-based training produces the same smoothing effect.

Scaling to high dimensions: manifold recovery

Real data like images live in enormous high-dimensional spaces, but only a tiny fraction of that space contains meaningful content. This meaningful region is called the data manifold , think of it as a crumpled sheet tucked inside a vast empty room. Generating a new image is really a problem of manifold recovery: inferring the shape of this hidden sheet from a finite number of training examples, then landing on a new point on it.

Score smoothing turns out to be directionally intelligent in high dimensions. It behaves differently depending on whether you are moving along the manifold (tangential direction) or toward it (normal direction):

• Tangential direction: Score smoothing slows particles down, preventing them from collapsing onto training points. This is where novelty comes from.
• Normal direction: The perfect score function is already smooth here (linear, if the manifold is flat), so smoothing doesn't interfere. Particles still converge efficiently onto the manifold.

The result is a balance: generated samples are both realistic (they reach the manifold) and novel (they don't just land on training points). Through theoretical analyses and numerical experiments, the work shows how score smoothing can enable the denoising dynamics to produce distributions on the relevant subspace without fully memorizing the training set.

What this means in practice

This is a theoretical paper, so there are no new model weights or training recipes to plug in directly. But the implications are significant for anyone building or studying generative models:

• Memorization is not a bug, it's an overfitting problem: If your diffusion model memorizes training data, it likely has too much capacity relative to the dataset size, or insufficient regularization.
• Weight decay is doing more than you think: Standard regularization is not just preventing overfitting in the usual sense , it is actively shaping the generative behavior of the model.
• New design directions open up: The results shed light on a fundamental mechanism behind the ability of neural network-based diffusion models to generalize and be creative, and motivate the exploration of alternative score estimators that facilitate generalization.
• Drug discovery and molecular generation: The interpolation effect means a model trained on known molecules doesn't just recall them , it explores the chemical space between them, potentially suggesting novel, chemically plausible candidates.

Limitations and what's next

The present work focuses on a vastly simplified setup compared to real-world scenarios. The theoretical results are proven for training data lying in a one-dimensional linear subspace , a far cry from the complexity of ImageNet or protein structure datasets. Extending the theory to arbitrarily spaced data, complex manifolds, and modern architectures like transformers remains open work.

The connections between score smoothing and the implicit bias of neural network training have also only been explored to a limited extent, especially in higher dimensions. But by establishing the causal chain , regularization causes score smoothing, score smoothing causes interpolation, interpolation causes creativity , the paper gives the field a concrete mechanism to build on and test.

Both the paper and the code for all numerical experiments are publicly available, making it straightforward to reproduce the figures and explore the theory on custom setups.