RGB MoCap Turns Phone Video Into 3D Animation Without Cloud Services
A free Windows app pipes video into editable 3D character animation locally, bundling six human-pose AI models with FBX, BVH, VMD and Blender export.
- RGB MoCap is a native Windows app wrapping six video-to-3D human pose models in one GUI.
- Supports ExMR, FlowHMR, GVHMR, MAMMA, PromptHMR and PEAR, with newly added NVIDIA GEM-X for commercial use.
- Exports to FBX, BVH, VMD for MMD, and portable Blender projects with retargeting to Mixamo, VRM, PMX.
- Contact correction reduces foot sliding; adaptive batching cut FlowHMR VRAM from 6.10 to 2.77 GiB.
- Full install is roughly 70.9 GB; requires 64-bit Windows, NVIDIA GPU, and OpenGL 4.3.
- Repo has crossed 1,150 stars and is MIT-licensed, though underlying model weights carry their own terms.
RGB MoCap is an open-source Windows application that converts ordinary color video, including phone clips, into 3D human motion. It packages recent pose-estimation models with a graphical interface, contact correction, character retargeting, and exports for Blender, Unity, VRM, and MikuMikuDance workflows.
The repository recently passed 1,150 GitHub stars and added NVIDIA GEM-X, whose license permits commercial use. Many academic pose networks and body models restrict commercial deployment, limiting their use in shipped games, paid animation work, and monetized VTuber projects.
Pose backends under one roof
The model documentation covers five offline backends, PEAR for experimental live preview, and the newer GEM-X integration. Each emphasizes a different combination of motion quality, hand tracking, memory use, or multi-person handling.
| Backend | Mode or role | Notable detail |
|---|---|---|
| ExMR | Offline | Produced better overall motion than PromptHMR in the maintainer’s subjective tests. |
| FlowHMR | Offline | Uses the SMPL-H body model and includes memory-saving attention changes. |
| GVHMR | Offline | Supports an optional HandFlow add-on for finger motion. |
| MAMMA | Offline | Handles multi-view reconstruction. |
| PromptHMR | Offline | Improved relative placement and interactions between multiple people in the maintainer’s tests, but does not generate finger motion. |
| PEAR | Live preview | Provides experimental low-latency pose visualization. |
| GEM-X | Additional estimator | Permits commercial use under its model license, with lower motion fidelity reported by the maintainer. |
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
