Runway's GWM Worlds 2 Streams Interactive AI Worlds at 720p in Real Time

Runway has released GWM Worlds 2, the next iteration of its General World Model line. The model generates interactive worlds in real time, streaming continuous 720p video at 24 fps and audio at 48,000 Hz, responding to user inputs as you explore. Rather than rendering a fixed clip, it keeps generating frame after frame, treating text actions and camera motion as live control signals.

The original GWM Worlds, shown in December, took a static scene and produced an immersive, infinite explorable space with geometry, lighting, and physics simulated in real time. Worlds 2 adds generated audio and substantially richer control over subjects and scenes.

WorldPrompt: scenes as persistent state plus event streams

The control format Runway calls WorldPrompt splits a world into what persists and what changes. The persistent side is a genesis prompt describing the environment, subjects with attributes and voices, and behavioral laws like gravity, collisions, and camera perspective, anchored by a first frame for visual grounding. The changing side is a timestamped event stream of free-form text actions, each addressed to a subject or the scene itself, with start and end times. Multiple actions can overlap, and camera input arrives as a separate per-frame stream of translation and rotation values.

A text action can carry a movement, a gesture, a line of dialogue, or a weather change. Speech is treated as another action carrying the line to be said. Because subjects are first-class entities in the schema, different actions can be addressed to different characters simultaneously, which is what enables multiplayer. The demo uses LiveKit to broadcast the stream and lets each client claim a role such as player 1, player 2, or director.

Autoregressive diffusion over video and audio

GWM Worlds 2 is an autoregressive diffusion model conditioned on three kinds of context at each step: a global context (the genesis prompt and first frame, always attended to); the current frame's inputs (camera pose plus any text actions whose timespan covers that frame); and past generated frames held in a sliding KV-cache window, with older frames evicted as the session advances.

Both the video and audio decoders are causal and cache-assisted. Every token attends to the global tokens, and each frame's video, text, and audio tokens causally attend to themselves and to past frames within the sliding window. That structure lets sessions run indefinitely without a preset length. The architecture builds on GWM-1, which was autoregressive, built on Gen-4.5, and controllable via camera pose and robot commands.

Three ways to drive it

Runway describes three usage modes for the same underlying model, trading latency for authorial control:

1. Ahead of time. You, optionally assisted by an LLM, author the full event stream upfront and the model renders it end to end. Suited to filmmaking and directing.
2. Turn-based. Generation runs until a decision point, you pick an action, an LLM formats it into the event stream, and generation continues. Suited to visual novels.
3. Real-time. The model streams continuously and reactions must land within tens of milliseconds, faster than typing or a VLM can respond. The demo handles this by binding keys and mouse buttons to premade prompts, so W might mean "the character moves forward" and left-click might mean "the character throws a ball."

Runway notes that ahead-of-time authoring still produces better quality because each prompt can describe much more of the scene. In real-time mode, the model must invent details on the fly with only the persistent context and previous frames as anchors.

Capabilities and honest limitations

Demonstrated capabilities include first- and third-person navigation, walking, driving, riding, flying, independent camera and subject control, object interactions, and spoken dialogue between named characters. Specific demos show a survivor jabbing a spear in a desert, a dirt bike carving through snow, a triceratops ride, and a rainy street conversation where speech, sweeping motions, and pointing gestures overlap on the timeline.

The limitations are documented plainly. Real-time generation trades fidelity for speed, so details, textures, and geometry can drift during quick camera movements. Long-term consistency is imperfect, and the model accepts no image references beyond the first frame. Fully leveraging free-form text control typically requires an external harness to track world state and generate actions dynamically, for example to drive NPC dialogue during player interaction.

Where this fits in the world model landscape

Google's Genie line, World Labs' Marble, and NVIDIA's Cosmos all occupy adjacent territory, and the competitive pressure on world models has intensified considerably in the past year. What Worlds 2 advances is the control surface. A cinematic generator can be judged on realism alone, but a world model must be evaluated on whether actions behave correctly across time, lighting, and occlusion. WorldPrompt makes that action space explicit and composable, with named subjects, behavioral laws, and overlapping timestamped events rather than a single text blob.

Runway positions the result as a foundation for interactive entertainment, virtual characters, robotics, and embodied-agent simulation. For policy training, a streaming simulator like this lets researchers roll out actions in a visually rich environment without deploying to hardware. For game and experience developers, the authoring flow can go from a one-line world description to a playable preset in seconds, with an LLM drafting the genesis prompt, first frame, and key bindings.

Access

GWM Worlds 2 is a research preview with no public pricing or self-serve access. The launch page includes a contact form for business inquiries. Runway describes the release as an early point on a curve it expects to steepen the way offline video generation did, moving from rough short clips toward production-grade output.