GLM Beats Langevin Sampling by 640 Hours on Medical Imaging Posteriors
A new latent variable trick turns messy Bayesian imaging posteriors into two-block Gibbs sampling that runs orders of magnitude faster than Langevin methods.
Takeaways
- New Gaussian Latent Machine unifies prior and posterior sampling for a broad class of Bayesian imaging models.
- Two-block Gibbs sampler reduces each step to a Gaussian solve plus m independent univariate draws.
- Handles Laplace, Student-t, symmetric Gamma, and GMM factors, plus isotropic total variation extensions.
- Converges in seconds on 96x96 image priors where MALA needs 640+ hours or fails outright.
- Learned GMM priors match natural image edge statistics and beat heavy-tailed priors on denoising and inpainting.
- Code repository at github.com/mkuric/GLM is announced but not yet released.
Sampling from complicated probability distributions is one of those quietly load-bearing problems in imaging. Every time you want a denoised MRI scan with a confidence interval, or a CT reconstruction that admits it might be wrong in a particular region, someone has to draw samples from a high-dimensional posterior. A new paper introduces the Gaussian Latent Machine (GLM), a latent variable construction that unifies a scattered family of prior sampling tricks and makes Gibbs sampling dramatically more efficient than the Langevin-based methods that currently dominate the field.
The setup starts from a very common shape of distribution called a product of experts, where the density factors as f(x) ∝ ∏ φᵢ((Kx)ᵢ). That single template covers a huge amount of Bayesian imaging: anisotropic total variation, fields-of-experts, sparse dictionary priors, total generalized variation, cosparse analysis models, and recent neural network priors all fit in, and so do the Gaussian and non-Gaussian likelihoods that show up in denoising, deblurring, computed tomography, and MRI.
Why Langevin sampling keeps stalling
In high dimensions, the go-to sampler is the Metropolis-adjusted Langevin algorithm (MALA), a discretization of an SDE that walks along the gradient of the log density with added Gaussian noise. It has strong convergence guarantees, but in practice it is fragile. Its step size has to be tuned carefully for every new problem, it struggles when the log density is nonsmooth or heavy-tailed, and when initialized in a low-gradient region it essentially does a random walk until it finds structure.
The authors demonstrate this brutally. In their largest image prior experiment, they did not even run MALA on 96×96 priors with GMM factors because it would have required more than 640 hours of runtime. For the same problems, their Gibbs sampler converged within about 200 seconds.