Marigold V2 turns an image editor into a one-step depth model

Marigold V2 retrains a large, pretrained image generator to estimate monocular depth, which maps the relative distance of each pixel from a single image. The release replaces Marigold’s Stable Diffusion U-Net with a diffusion transformer, reduces inference to one denoising pass, and reports benchmark-leading results while processing 2048×2048 images within 29 GB of GPU memory.

Researchers from HUAWEI Bayer Lab, EPFL, and the University of Bologna trained the model in less than a week on one 32 GB GPU. The project repository includes Apache 2.0-licensed code, while the paper is slated for ACM Transactions on Graphics and SIGGRAPH Asia. The team also published model weights and a browser demo.

A DiT learns depth on one GPU

The model repurposes Qwen-Image-Edit-2509, an open-source image-editing diffusion transformer, for dense prediction. Its training setup quantizes the backbone to 4-bit precision and adds rank-128 QLoRA adapters, which update a small set of low-rank parameters while leaving the quantized backbone frozen.

This configuration reduces memory use during training and deployment. Competing detail-oriented depth systems often require 80 GB accelerators, distributed training, or several diffusion steps per image. Marigold V2 uses one inference step, handles 2K inputs without running out of memory, and supports multiple tasks by loading different LoRA adapters over the same backbone.

Loss functions built for noisy labels

Synthetic datasets such as Hypersim provide dense depth labels, but their boundaries can be unreliable. Around grass, hair, railings, and other thin structures, adjacent pixels may alternate between foreground and background depths. Pixel-wise L1 or mean squared error then penalizes visually accurate predictions that disagree with flawed labels.