Meta and Korea University's DecHOI Fixes AI Characters Fumbling Object Interactions
A new framework splits human-object interaction synthesis into two specialized networks, removing the need for hand-specified waypoints and cutting penetration errors on standard benchmarks.
ProTakeaways
- DecHOI splits human-object interaction into a trajectory generator and an action generator instead of one network.
- Removes the need for manually specified intermediate waypoints, requiring only start pose, goal, text, and object geometry.
- Compact discriminator targets hand and foot dynamics to reduce object penetration and mid-air hovering artifacts.
- Achieves FID 0.33 on FullBodyManipulation, versus 1.58 for CHOIS and 2.06 for HOIFHLI.
- DynaPlan module adds A*-based re-planning around a predicted moving obstacle using Social-STGCNN forecasts.
- Limited to rigid objects, single obstacle scenarios, and no public code release is currently listed.
Getting a virtual character to pick up a chair and carry it across a room sounds like a solved problem, but current diffusion-based systems still produce hands that clip through boxes, feet that hover above the floor, and objects that drift out of sync with the body holding them. A team from Korea University, Meta, and UT Austin has proposed DecHOI, a framework that tackles these artifacts by refusing to solve the whole problem in one network.
Why one model keeps fumbling the handoff
Prior systems such as CHOIS and HOIFHLI generate 4D human-object motion by conditioning a diffusion model on text plus a sequence of waypoints. At inference time, the user must manually provide not only the start and goal points but also intermediate waypoints, which narrows the model's generative scope and reduces autonomy. Human-object interaction also requires resolving the motion of both the person and the object in every frame, and assigning all objectives to a single network elevates optimization complexity and risks practical intractability. The visible symptoms: objects that hover in mid-air, hands that penetrate meshes, and motion that desyncs from the item being manipulated.
Two experts instead of one generalist
DecHOI decouples the pipeline into a Trajectory Generator (TG) and an Action Generator (AG). The TG plans where the human and object should go; the AG then fills in the detailed joint motion along those paths. Partitioning the model into two lightweight expert networks reduces the optimization complexity for each branch and mitigates unsynchronized interactions.