KCL's HapTile Teaches Robots to Feel, Boosting Peg Insertion from 0% to 90%
Researchers from King's College London introduce HapTile, a 1,726-demo dataset that lets robots feel contact during teleoperation — lifting peg insertion from 0% to 90% success
Takeaways
- New dataset: Researchers at King's College London release HapTile, 1,726 robot manipulation demos with synchronized vision, fingertip tactile, language, and action data.
- Haptic teleoperation: First dataset where the human operator receives live vibration feedback from the robot's fingertip sensors during data collection.
- Dramatic gains: Adding tactile input lifts π₀'s peg insertion success from 0% to 90%; whiteboard wiping jumps from 50% to 100%.
- Two tactile modes: Raw tactile images help contact localization tasks; marker-tracked displacement features help shear/force tasks — neither dominates universally.
- Open and reproducible: Built on UR5e + Robotiq gripper + Meta Quest; data in LeRobot/OpenPI format at haptile-dataset.github.io.
- Limitations: Single lab scene, binary haptic feedback only, and large VLAs like π₀ need more data to fully exploit the tactile channel.
Most robot learning datasets teach robots to see. HapTile, a new dataset from researchers at King's College London, Huawei Noah's Ark Lab, and University College London, bets that touch is just as important. The paper, led by Shan Luo's group at KCL, introduces the first manipulation dataset that closes the loop in both directions: fingertip tactile sensors on the robot, and live haptic vibration fed back to the human teleoperator in real time.
The gap nobody was filling
Vision-Language-Action (VLA) models , systems that take camera images and a language instruction and output robot actions , have become the dominant paradigm for general-purpose manipulation. Models like RT-2, OpenVLA, and π₀ show impressive generalization, but they share a common blind spot: they cannot feel. Tasks like peg insertion, cloth folding, or wiping a surface all involve contact forces and slip that a camera simply cannot see, especially when the gripper occludes the interaction.
Tactile datasets do exist, but they each miss at least one critical ingredient. Some have touch data but no action trajectories. Others have actions but use wrist-level force-torque sensors that measure bulk load rather than localized fingertip contact. And nearly all were collected with teleoperation pipelines where the human operator is flying blind , no feedback about whether the robot is actually gripping something or about to drop it.

What HapTile actually contains
HapTile is a visuotactile, contact-grounded manipulation dataset with 1,726 demonstrations across 38 tasks, collected by 9 human operators via a haptic-informed teleoperation interface. The skills span everyday manipulation:
- Pick-and-place, stacking, and removing objects
- Folding (including origami tasks)