Patch's Anymove Turns AI Video Into 3D Creature Motion for $17

Anymove extracts 3D motion capture from a single AI-generated video using a dual-camera trick and a DINOv2 joint reader, outputting rigged GLB animations for about $3 per move.

·
·
·
Patch's Anymove Turns AI Video Into 3D Creature Motion for $17PRO
Read2 min
TypeRepo
  • Patch released anymove, an Apache-2.0 pipeline turning one AI video into 3D mocap for any creature.
  • Two cameras rendered in one frame let joints be triangulated without explicit calibration.
  • A DINOv2 backbone is fine-tuned per creature on 2,200 synthetic poses in 20 minutes.
  • Costs about $3 per usable move and $17 for a new creature with 5 moves on fal.
  • Works for roars, slams, takeoffs, walks and runs on two or four legs; fails on spider walks and bird wings.
  • Ships as a Claude Code plugin with per-stage agent skills and GLB output at 24 fps.

Patch has released anymove, an open-source motion-capture pipeline that converts AI-generated split-screen video into 3D animation for custom creature rigs. Each clip shows the same character from the front and side at once. A per-creature detector locates joints in both views, the software reconstructs their 3D positions and fits a skeleton, and the pipeline exports a rigged GLB file at 24 frames per second.

Within its first few weeks, the repository collected more than 600 GitHub stars. Patch estimates roughly $3 in generation costs for each accepted move. Creating a creature with about five moves costs roughly $17 and takes a few hours, although retries, GPU choice and optional LoRA training can raise that figure.

Two views recover one skeleton

The generated clip places a front view on the left side of the frame and a side view on the right. The repository credits the original method to @CanItCode and a MiniMax H3 LoRA refinement to @Graalitoo. A joint detector tracks the creature in both views, after which the solver reconstructs a 3D target for every joint and fits the rig frame by frame.

When the generated layout preserves a shared vertical axis, each joint should occupy the same pixel row in both halves of the frame. The front view supplies horizontal position and height; the side view supplies depth and a second height estimate. Combining those measurements recovers the joint’s 3D position under the pipeline’s fixed-camera assumptions, avoiding the camera-tracking stage required for footage shot with moving or unknown cameras.

Front and side cameras positioned around a rigged dragon
The pipeline asks the video model to preserve front and side views in one frame.

One detector learns one creature

Anymove builds each joint detector on Meta’s DINOv2 vision backbone and trains it for one creature. Synthetic renders provide thousands of random poses with known joint coordinates, allowing the model to learn a dragon, wolf or other custom body plan without a manually labeled video dataset. Patch reports that average joint error falls from about 40 pixels to 6 pixels during training.

Pro article

This story is for Pro members

You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.

Trending
  • No trending articles

Comments

avatar

Next Reads