Skip to main content
Back

Georgia Tech's FiLM-Nav Teaches Robots to Navigate Any Room 20% More Efficiently

Georgia Tech researchers fine-tune a Mamba-based vision-language model as a frontier picker, topping HM3D ObjectNav and HM3D-OVON on path efficiency.

Paper, Full Access
  • FiLM-Nav fine-tunes a 2.8B Mamba-based VLM (Cobra) directly as a frontier-selection navigation policy.
  • Sets state-of-the-art 41.3% SPL and 77.0% SR on HM3D-v0.2 ObjectNav, beating Uni-NaVid.
  • Achieves best SPL across all three splits of the HM3D-OVON open-vocabulary benchmark.
  • Training mix combines ObjectNav, OVON, ImageNav and an auxiliary spatial reasoning task.
  • Mamba backbone gives 60% faster inference and constant memory versus transformer VLMs.
  • Limitations: 2D mapping cannot handle stairs, single-object goals only, no physical robot deployment yet.

Getting a robot to find the "gravity chair" in a house it has never seen sounds simple until you try it. The robot needs to understand what a gravity chair is, remember which rooms it has already walked through, and decide which unexplored doorway to try next. A new paper called FiLM-Nav from Georgia Tech tackles this by taking a pre-trained vision-language model and fine-tuning it directly as the navigation policy, rather than using it as a bolt-on reasoning module.

FiLM-Nav sets a new state of the art on both the HM3D ObjectNav and the harder HM3D-OVON open-vocabulary benchmark, especially on path efficiency, the metric that matters when a robot has finite battery.

Why plugging a VLM into a robot is harder than it looks

Object Goal Navigation (ObjectNav) drops an agent in an unseen 3D environment and tells it, in language, to go find something. The open-vocabulary version, OVON, ups the difficulty by allowing any object description in free-form language, rather than a fixed list of six categories.

Existing approaches often use foundation models for high-level planning, instruction interpretation, reasoning about object locations from semantic cues, or enriching environmental representations. Because these models are typically trained on disembodied web data, they exhibit weaknesses in spatial-temporal reasoning and physical grounding, both critical for efficient robotic task execution. Common pipelines pass a text summary of what the camera saw into an LLM, ask for a plan, and lose most of the visual detail in the translation.