Alibaba's MiniMax-H3-Fun-Controlnet-Union Steers Video Generation With One 6.8 GB Checkpoint
Alibaba PAI drops a single ControlNet-Union checkpoint that adds Canny, Depth, HED, MLSD, Pose and inpainting to MiniMax-H3 video generation.
PRO- Alibaba PAI released MiniMax-H3-Fun-Controlnet-Union, adding structural control to the MiniMax-H3 video model.
- One 6.81 GB checkpoint handles Canny, Depth, HED, MLSD, Pose and video inpainting with no swapping.
- Control branch attaches to just 5 of 50 transformer blocks (layers 0, 10, 20, 30, 40) via zero-gated projection.
- Guidance-distilled: keep guidance_scale at 1.0, one forward pass per step, no CFG tax.
- Runs via the VideoX-Fun pipeline, needs 80 GB GPU with offloading.
- Outputs cap at 15 seconds, 24 fps, following the control video's aspect ratio.
Alibaba's PAI team just plugged a big hole in the MiniMax-H3 video model ecosystem. They released MiniMax-H3-Fun-Controlnet-Union, a single 6.81 GB safetensors checkpoint that steers H3 video generation with edge maps, depth maps, human poses, and more, without swapping models between conditions.
If you have been using MiniMax-H3 for text-to-video or image-to-video and wanted the kind of structural control that ControlNet gave you on image models, this is the missing piece.
One checkpoint, five control signals
The headline feature is the Union design. A single checkpoint conditions the MiniMax-H3 video generator on Canny, Depth, HED, MLSD, or Pose control videos, and also runs video inpainting, with no per-condition checkpoint switching. That saves a lot of disk and VRAM juggling compared to shipping one ControlNet per modality.
The supported control signals cover most of what people actually reach for in video work:
- Canny for hard edge outlines from a reference clip
- Depth for preserving 3D scene structure and camera motion
- HED for softer, painterly edge guidance
- MLSD for straight line and architectural structure
- Pose for driving characters with human skeleton keypoints
- Inpainting for masked region regeneration inside a video
How the control branch actually attaches
Rather than duplicating the whole transformer like the original image ControlNets did, this one is surgical. The control branch attaches to 5 of the 50 transformer blocks (layers 0, 10, 20, 30, 40), and every control skip is added to the main branch through a zero-gated projection. That is why the checkpoint is only 6.8 GB instead of a full copy of the base weights.
The zero-gated projection is the classic ControlNet trick, where the injection layers start at zero so the base model behaves normally at initialization and gradually learns to accept guidance during training. Here it is applied at just five evenly spaced depths through the 50-block stack.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
