FLUX 3 Action brings joint video prediction to robot control

Black Forest Labs has expanded its FLUX model family into robotics with FLUX 3 Action, a 7-billion-parameter, open-weight World Action Model. Given camera frames, robot state and a text instruction, it predicts both the robot’s next actions and the resulting scene in one forward pass.

The release includes model weights, code, fine-tuning recipes and benchmarks on Hugging Face. Black Forest Labs also worked with NVIDIA on native LeRobot integration and support for deployment on Jetson edge hardware.

A smaller route to joint prediction

Open robot policies commonly use one of two architectures. World Action Models, or WAMs, generate future video alongside actions, giving the policy a representation of how the scene may change. Vision Language Action models, or VLAs, map visual observations and instructions directly to actions, which usually reduces inference cost.

BFL’s published comparisons show the cost of joint prediction in existing systems. On an NVIDIA B200 GPU, leading WAMs required about seven times more inference compute for each second of robot motion than faster action-only policies.

FLUX 3 Action uses a smaller backbone and distillation to narrow that gap. On RoboLab, a robot-manipulation benchmark, its single-step checkpoint recorded a 38.3% success rate. Cosmos 3 Nano reached 36.8% in the same evaluation. BFL also reports that FLUX 3 Action processed robot motion 1.45 to 1.66 times faster than pi0.5, while predicting a 2.13-second action horizon compared with 1 second for pi0.5.

A guidance-distilled checkpoint increased RoboLab success to 42.2%. Its reported throughput was 2.85 to 3.15 times that of Cosmos 3 Nano, although the variant trades some of the single-step model’s latency advantage for higher task success.

One pass, two outputs

FLUX 3 Action receives recent camera frames, the robot’s current joint state and a natural-language task description. It returns 32 actions plus predicted frames showing how the scene should evolve.

The controller uses a receding-horizon loop: the robot executes part of the proposed action sequence, captures another observation and plans again. Repeated observation allows the policy to adjust after an object shifts, a grasp misses or the arm deviates from the predicted path.

Architecturally, the model reuses the FLUX 3 multimodal backbone. Text, video and robot-state tokens pass through a shared transformer, after which separate decoder heads produce future frames and robot actions. This design keeps visual prediction and control within the same learned representation.

BFL attributes the model’s efficiency to two changes. Multimodal Self-Flow pretraining supports a backbone less than half the size of Cosmos 3 Nano, while distillation compresses the inference procedure. Guidance distillation approximates classifier-free guidance without separate conditional and unconditional passes. Step distillation reduces four iterative sampling steps to one.

Training across robots, hands and games

The initial pretraining stage followed the FLUX 3 recipe and combined image, video and audio data. Video supplied more than 95% of its training tokens. An action-focused midtraining stage then mixed that corpus with demonstrations from games, hand tracking, handheld grippers and robot teleoperation.

Action labels use two standardized representations. A 64-dimensional gaming space encodes mouse movement, mouse buttons and keyboard input. A 50-dimensional EE50 pose space represents two end effectors for hand, gripper and robot control. These shared formats let one model learn from several kinds of embodied interaction.

During fine-tuning on DROID, a large dataset of robot-manipulation demonstrations, the midtraining weights remain frozen for the first 1,000 steps. The learning rate then warms linearly to 2e-4 over 2,000 steps. That schedule limits unstable gradients from the newly initialized action heads before they update the pretrained backbone.

Where the latency savings come from

Three optimizations account for most of the deployment gains. Guidance distillation removes the second forward pass used by classifier-free guidance, step distillation collapses four sampling iterations into one, and FP8 execution with torch.compile reduces arithmetic and runtime overhead.

BFL reports that row-wise dynamic FP8 produces more than a 1.4-times speedup on local GPUs, including RTX PRO hardware and the RTX 5090, with negligible change in measured task success. Actual throughput will depend on batch size, observation resolution, action horizon, compiler support and the target GPU.

A 30-run test on a Franka arm

Positronic Robotics evaluated FLUX 3 Action on a Franka arm using ten DROID tasks and three attempts per task. Cosmos 3 Nano, DreamZero and pi0.5 received the same tasks, physical setup, 240-second time limit and scoring procedure. The operator was blinded to the policy used for each attempt.

The evaluation contains only 30 attempts per model and came from a release partner. Independent tests across more tasks, robot platforms and environmental variation will be needed to establish how reliably the ranking generalizes.

Pairing reflexes with a reasoner

Long-horizon tasks can require planning beyond the action model’s prediction window. BFL tested a hybrid controller in which a reasoning model called GPT 6 Astra supervised FLUX 3 Action. The supervisor could accept the proposed trajectory, modify selected actions or take direct control.

Within BFL’s comparison set, the hybrid configuration was 29% cheaper and 40% faster than the strongest alternative configuration. The reported costs depend on model pricing, hardware, task duration and how often the supervisor intervenes.

A capable low-level policy reduces the number and duration of reasoning-model calls. That division of labor gives the fast action model routine control while reserving the more expensive model for planning, correction and recovery.

Three downloadable checkpoints

• flux-3-action-base: the adaptation checkpoint with shared, frozen video and text encoders.
• flux-3-action-so101: an SO-101 LeRobot policy with a task-specific LoRA recipe for parameter-efficient fine-tuning.
• flux-3-action-droid: the DROID policy, with optional guidance-distilled and step-distilled variants.

From checkpoints to a robot

1. Select the checkpoint closest to the target robot and task.
2. Collect demonstrations using the same observations and action representation expected at inference.
3. Fine-tune the policy, using LoRA when limited memory or rapid task adaptation matters.
4. Measure task success, control frequency and recovery behavior on held-out scenarios.
5. Deploy through LeRobot or the supported Jetson path, with robot-specific limits and monitoring.

Commercial and production use still requires review of each repository’s license and model card. Physical deployments also need independent safety controls, including workspace limits, collision handling, emergency stops and validation against unexpected observations.

Games and drones widen the target

Gaming recordings give FLUX 3 Action experience with environments where an agent observes pixels and chooses structured actions. BFL also presents early task-specific results for game playing and simulated drone control.

Games support parallel simulation, faster-than-real-time execution and repeatable evaluation without physical hardware. Those properties make them useful for testing action prediction, long-horizon planning and coordination between a fast policy and a slower reasoning model.

The same pixel-to-action interface could eventually support software control, including mouse and keyboard interaction. Current evidence covers selected games, simulation and robot manipulation; reliable transfer to general computer-use agents remains unproven.

Claims that still need testing

FLUX 3 Action shows how a smaller backbone, joint video-action training and aggressive distillation can reduce the historical inference cost of World Action Models. The published results combine competitive benchmark accuracy, longer action horizons, faster inference and a practical fine-tuning path.

The remaining questions concern generalization and reproducibility: performance on unseen objects and environments, stability over long deployments, latency on specific Jetson devices, data requirements for new embodiments and results from independent evaluators. Those measurements will determine where the model fits beyond BFL’s reported benchmarks.