---
title: "WPI Strips Robot Arm Control Down to a Single Camera Using Inpainting"
subtitle: "Researchers used image inpainting to erase training markers and reconstruct occluded robot parts, enabling markerless visual control without camera calibration or kinematic models."
url: "https://alphasignal.ai/news/wpi-strips-robot-arm-control-down-to-a-single-camera-using-inpainting"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-14T00:00:00.000Z"
updated: "2026-08-23T21:37:44.157Z"
type: "Paper"
topics:
  - "Robotics"
  - "Image"
subtopics:
  - "Vla Models"
  - "Pose Estimation"
  - "Image Editing"
upvotes: 1544
source_url: "https://arxiv.org/abs/2604.13309"
comments: 0
truncated: true
---

# WPI Strips Robot Arm Control Down to a Single Camera Using Inpainting

> Researchers used image inpainting to erase training markers and reconstruct occluded robot parts, enabling markerless visual control without camera calibration or kinematic models.

Controlling a robot arm by watching it move sounds simple, but most vision-based control pipelines lean heavily on fiducial markers, precise camera calibration, and accurate kinematic models. A team from Worcester Polytechnic Institute has shown you can strip all of that away by using image inpainting in two clever places: once to erase training markers from labeled data, and again at runtime to reconstruct parts of the robot hidden by obstacles.

The [paper](https://arxiv.org/abs/2604.13309) presents a visual servoing framework that controls a Franka Emika Panda arm using only natural keypoints on the robot's body, detected from a single fixed RGB camera. The authors claim this is the first eye-to-hand image-based visual servoing (IBVS) pipeline to achieve 3D motion control from one camera and natural 2D keypoints, without depth sensing or explicit robot or camera models.

## Why markers are a bottleneck

Traditional IBVS depends on ArUco tags, AprilTags, or QR codes stuck to the robot. These fail under self-occlusion, motion blur, reflections, or bad lighting, and attaching them is impractical in many real-world applications. Borrowing techniques from human pose estimation runs into a data problem: there is no COCO-scale dataset of annotated robot keypoints.

Earlier attempts either trained in simulation (creating a sim-to-real gap) or leaned on camera intrinsics plus forward kinematics to project 3D joint frames into image space. That works for rigid arms with accurate models, but breaks for soft, underactuated, or cheaply built robots where the kinematic chain can't be trusted.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Utilizing Inpainting for Keypoint Detection for Vision-Based Control of Robotic Manipulators](https://arxiv.org/abs/2604.13309)
- [KPDataGenerator GitHub Repository](https://github.com/Jani-C/KPDataGenerator)
- [LaMa: Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://github.com/advimman/lama)
- [Full paper HTML on arXiv](https://arxiv.org/html/2604.13309)