Skip to main content
Back

Amsterdam and Mila Prove AI Can Recover Hidden Causal Factors Without Clean Probability

A new AISTATS 2026 result extends causal representation learning to broken density functions, using sparsity and Gaussianity to recover hidden variables.

Paper, Full Access
  • New AISTATS 2026 paper proves identifiability of causal latents even when their density is degenerate.
  • Setting: Gaussian mixture latents observed through piecewise affine mixing, i.e. ReLU networks.
  • Sparsity regularization replaces density analyticity, yielding recovery up to permutation and scaling.
  • Two-stage estimator: enforce sparsity, then enforce Gaussianity on learned representation.
  • Experiments on synthetic and ball image data show near-block-diagonal recovery of true factors.
  • Related open code from same authors: github.com/danrux/sparsity-crl.

Causal representation learning has a long-running assumption problem. Most theoretical guarantees for pulling clean latent variables out of tangled observations quietly require the latents to have a nice, well-defined probability density. A new paper from the University of Amsterdam and Mila breaks that assumption on purpose, and shows you can still recover the hidden factors anyway.

The work, Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing, was accepted to AISTATS 2026. It targets a setting where latent variables follow a potentially degenerate Gaussian mixture distribution and are observed only through a piecewise affine mixing function. That is exactly the kind of structure you get from a ReLU or Leaky ReLU network sitting on top of clustered, correlated latent factors.

When the density function falls apart

Causal representation learning (CRL) tries to invert the data-generating process: given high-dimensional observations like images, recover the small set of underlying variables that generated them, even when those variables are causally related.

Most existing identifiability proofs lean on a technical property: the probability density of the latents has to be analytic. Prior results on Gaussian mixture latents with piecewise affine mixing, like Kivva et al. 2022, exploit exactly that. Real representations are often degenerate, though. Sparse latents, low-rank clusters, or components that live on a lower-dimensional subspace all produce Gaussians whose covariance is not full rank, which means the density blows up or is simply not defined. Analyticity arguments break, and the classical proofs no longer apply.