Open-Source FAH Tracks Hands in Real Time at 65fps With Stunning Accuracy

FAH, an open-source family of monocular hand estimators from Jinkun Cao, reverses a common hand-tracking pipeline. Instead of predicting the parameters of a hand rig such as MANO and projecting its mesh into the image, FAH predicts 2D landmarks, root-relative 3D joints and dense surface points, then fits a parametric mesh to those observations.

The point-first design reduces the alignment errors introduced when a fixed rig cannot express the hand visible to a single RGB camera. According to the paper, FAH achieves state-of-the-art alignment on standard benchmarks while running in real time on a consumer GPU. The release includes four model sizes, TensorRT engines and optional outputs for anatomical parts, contact probability, temporal refinement and world-space placement.

The rig becomes the bottleneck

MANO compresses hand shape and pose into a small parameter vector that controls a fixed mesh topology. Every valid vector produces a coherent, riggable hand, but the representation constrains what the network can place at a given image coordinate. Errors in the estimated parameters propagate to projected joints, vertices and later processing stages.

Cao reports that directly predicted 3D joints agree more closely with 2D observations than joints derived from regressed rig parameters. Fitting MANO or MHR-Hand after point prediction preserves image-space alignment while still producing a watertight surface with a standard topology. The fitted mesh can then feed renderers, animation systems and geometry-based analysis.