Tsinghua's WorldFly Teaches Drones to Imagine Around Corners Before Flying
WorldFly teaches drones to imagine their path forward, using a world model to predict future video frames and navigation actions simultaneously in dense urban environments.
- WorldFly is a new VLA framework for drone navigation that jointly predicts future video frames and flight actions using a dual-branch coupled flow matching architecture.
- The key insight: drones navigating dense urban environments need to imagine future states, not just react to current observations, to handle occlusions and sharp turns.
- The authors introduce the Urban Canyon Traversal Benchmark, a new evaluation suite specifically targeting severe occlusions and drastic viewpoint transitions.
- WorldFly outperforms baselines including OpenFly-Agent, with the largest gains on hard scenarios and unseen environments.
- The dual-branch architecture uses cross-attention to couple video prediction and action prediction, making them mutually consistent during training and inference.
- No code or model weights have been publicly released alongside the paper.
Most autonomous drone navigation systems react to what they can see right now. That works fine in open skies, but in dense urban environments -- tight corridors between buildings, sharp corners, sudden occlusions -- a purely reactive approach breaks down fast. A new paper from researchers at Tsinghua University and Ryerson University proposes a different strategy: teach the drone to imagine what it is about to see before it gets there.
The result is WorldFly, a Vision-Language-Action (VLA) framework that couples a world model directly into the navigation policy. Rather than mapping observations to actions in one step, WorldFly jointly generates a prediction of future video frames and the corresponding flight actions at the same time, using each to guide the other.
Why drones struggle in cities
Extending the VLA paradigm to aerial platforms introduces fundamental challenges absent in ground-based settings. UAVs operate in a fully dynamic 6-DoF state space subject to continuous gravitational and inertial forces, where pitch and roll dynamics are tightly coupled with translational motion. Add buildings into the mix and the problem compounds: a drone rounding a corner between two skyscrapers loses visual context almost instantly.
Current vision-language navigation research for UAVs relies on detailed, pre-specified instructions to guide the UAV along predetermined routes. However, real-world outdoor exploration typically occurs in unknown environments where detailed navigation instructions are unavailable. Instead, only coarse-grained positional or directional guidance can be provided, requiring UAVs to autonomously navigate through continuous planning and obstacle avoidance.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
