Hunan University's SinRef-6D Teaches Robots to Grab Unseen Objects From One Photo
SinRef-6D estimates the 6-DoF pose of unseen objects from a single RGB-D reference view, powering a robotic grasping system with 85% success.
PRO- SinRef-6D estimates 6-DoF absolute pose of unseen objects from just one RGB-D reference view.
- Uses Mamba-style state space models for linear-complexity spatial feature extraction on points and images.
- Two unshared GeoTransformers separate coarse alignment from fine refinement, boosting AR from 36.5 to 62.2.
- Hits 90.3% ADD-0.1d on LineMod and 64.6% mean AR across five BOP datasets, CAD-free.
- Real-world robotic grasping achieves 85% success on planar and 74% on non-planar placements.
- Code, demos, and integrated robotic system at project page.
Getting a robot to pick up something it has never seen before is a surprisingly hard problem. Most pose estimation methods either need a textured CAD model of the object, or a dense set of reference views (often 16 to 200 of them) to reconstruct a 3D representation. Both are painful to acquire at scale, which is exactly why robotics labs have spent years chasing something lighter.
A team from Hunan University, Nanyang Technological University, and collaborators has pushed that frontier hard with SinRef-6D, a framework that estimates the full 6-DoF absolute pose of an unseen object from a single pose-labeled RGB-D reference image. Once pre-trained on synthetic data, the model generalizes to new objects using only that lone reference view, and the authors wrap it in a full hardware plus software grasping system to prove it holds up in the real world.
Why one reference view breaks everything
Cutting down from 200 reference views to 1 sounds like a nice engineering optimization until you realize what breaks. With a single view, there are large pose discrepancies between the reference and any query view of the object, and there is very limited geometric and spatial information to work with. Template matching pipelines fall apart because you no longer have full angular coverage, and 3D reconstruction methods like Structure-from-Motion or NeRF-style rebuilds simply cannot run.
Prior single-reference work (One2Any, Any6D, UNOPose) mostly targets relative pose between two views, which is not enough for a robot arm. Manipulation requires the object's absolute pose in the camera coordinate system before any grasp can be planned. SinRef-6D is designed specifically for that manipulation-oriented, absolute-pose setting.
Iterative point-wise alignment in object space
Rather than matching templates or regressing a pose directly, SinRef-6D formulates the problem as point cloud alignment inside the object's own coordinate frame. The reference point cloud gets transformed into object space using the known reference pose. The query point cloud gets transformed with the current pose estimate, refined iteratively.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
