Tsinghua's OVAL Gives Robots Persistent Memory to Find Any Object
A new framework for robot navigation stores what it sees across many tasks, letting an agent chase a stream of open-ended goals without forgetting.
ProTakeaways
- OVAL is a lifelong open-vocabulary ObjectNav framework that reuses memory across sequential goals in unseen scenes.
- Stores objects via label, image buffer, 3D coordinate, HSV scene histogram, and a view-quality confidence score.
- De-duplicates instances with scene and position similarity, falling back to SuperGlue feature matching in ambiguous cases.
- New frontier policy scores candidates by distance, semantic co-occurrence with the goal, and a footprint penalty.
- Hits 68.1 SR on HM3D and 44.1 SR on MP3D lifelong ObjectNav, beating VLFM and GOAT baselines.
- Runs at 1.14 FPS on an RTX 4090D using GPT-4o for verification and Grounded-SAM2 for auto-labeling.
A team from Tsinghua University's Shenzhen International Graduate School has released OVAL, a framework that lets a mobile robot handle a stream of open-vocabulary object goals in an unseen environment without wiping its memory between tasks. Rather than finding a cup, forgetting the room, then starting from scratch to locate a remote, the agent builds up a structured, persistent memory it can query for whatever comes next.
Why single-shot ObjectNav breaks down in real homes
Object Goal Navigation (ObjectNav) is defined as navigating an unseen environment to find a target object, and it underpins most household and service robot demos. Almost every published method assumes one goal per episode. Real deployments rarely involve a single isolated objective, and instead require sequentially achieving lifelong open-vocabulary goals, like being asked to find a cup and then, minutes later, a remote control in a completely different room.
Two things tend to collapse when you extend existing systems to this lifelong setting:
- Under open-vocabulary auto-label settings, directly storing raw, unpredictable textual labels in memory is impractical, leading to duplicate entries, synonym confusion, and unpredictable misrecognition.
- Prior memory-based systems like GOAT, which employ Mask R-CNN as an open-vocabulary detector, require predefined vocabularies, so they cannot retain information about previously encountered objects when new categories are added.
Once you go truly open-vocabulary, the memory index itself becomes noisy and the frontier-exploration policy stops making good use of what has been seen.