The real world is vast, and in order for physical AI systems to operate within it, they must understand how the scene is changing, predict what will happen next, and determine how their actions will affect the world.
Machines operate at the edge in factories, warehouses, hospitals, and more. Understanding and acting in these environments requires models that provide datacenter-level performance on memory-constrained systems.
Today, we are releasing NVIDIA Cosmos 3 Edge in Hugging Face’s Cosmos 3 repository. It is a 4 billion-parameter open-world model that helps robots and vision AI agents understand their environment, reason in real-time, and generate robot actions on edge devices.
Download the model: https://huggingface.co/nvidia/Cosmos3-Edge
This model delivers memory-efficient, high-throughput inference across NVIDIA edge computers, including NVIDIA Jetson, including NVIDIA RTX PRO GPUs, NVIDIA DGX, NVIDIA GeForce RTX™ GPUs, and the newly announced Jetson T2000 and T3000 modules.
It is designed as a compact open model that can function as a small vision language model (VLM) with best-in-class throughput and accuracy with real-time inference.
Cosmos 3 Edge operates as a trained world action model (WAM) at robot control resolution (640×360 observations) and provides real-time inference on NVIDIA Jetson Thor, generating 32 actions per inference while achieving real-time control at 15 Hz.
Among models of similar size (parameter 4B), Cosmos 3 Edge ranks #1 on the state-of-the-art VANTAGE-Bench for vision analysis and robot policy learning, establishing the state-of-the-art in smart infrastructure and robotics.
What is world modeling?
World models learn how the environment changes over time. Represents objects, movements, spatial relationships, and effects of actions.
Consider a robot that reaches for an object. Recognizing objects is only the first step. The robot also needs to understand where the object is, how its grippers are moving, what happens when contact occurs, and which actions are most likely to successfully complete the task.
The world model helps the robot reason about these relationships. You can predict the visual consequences of an action, infer which actions caused a change, and generate actions that produce desired results.
Cosmos 3 Edge brings these features together in one model on the device. That shared representation allows physical AI systems to understand the current state of the world, simulate possible futures, and connect those futures to actions.
Two substation towers, one shared representation
Cosmos 3 combines two transformer towers.
Autoregressive towers: Processing vision and text tokens for comprehension and inference. Diffusion Tower: Process vision, audio, and action tokens for prediction, generation, and neural simulation.
The two towers maintain separate normalization layers and multilayer perceptrons. They share a multimodal attention layer that coordinates information across language, video, audio, and actions.
This design allows the model to reason about the scene before producing output. Depending on the task, Cosmos 3 can generate inference tokens from autoregressive towers or denoised video and action tokens from diffusion towers.
Attention patterns are also adapted for each type of information. Language uses causal attention, where each token pays attention to the tokens that came before it. Diffusion tokens support a wide range of available contexts and consistent prediction and generation.
Together, these components give the model a common representation of what is happening, what will happen next, and how actions affect outcomes.
General expression of action
Physical systems describe actions differently. Vehicles may express actions through ego poses and movements. The camera uses camera motion. Robotic arms use end effector poses, but hands and grippers also need to represent grasping states.
Cosmos 3 maps these different embodiments to a common action representation.
Actions are encoded as compact geometric vectors that capture:
Translational rotation operation state
This creates a direct link between control and the visual structure of the world. The model can relate pixel changes to physical movement, spatial relationships, and control inputs.
As a result, the resulting video is more than just a visual prediction. It can represent how the world is expected to change in response to an action. This provides developers with training data based on movement, control, and cause and effect.
Policy mode
As a policy, Cosmos 3 predicts actions with expected visual consequences.
Models can generate what the system should do and simulate what is likely to happen next. This connects world modeling directly to robot policy training and evaluation.
Current state is input, actions and visual results are output.
These modes allow a single model to learn cause, effect, and policy. Because actions can flow in both directions through the model, Cosmos 3 Edge can predict the effect of an action and infer an action from its effect.
Instructions: Pick up the banana and place it on a plate.
We are also releasing Cosmos 3 Edge Policy (DROID). This is a robot manipulation policy that is post-trained on the DROID dataset for pick-and-place tasks and also comes with a post-training script.
Using small clusters of H100 or NVIDIA DGX Station, developers can efficiently fine-tune Cosmos 3 Edge to their target workloads before deploying to NVIDIA edge and accelerated computing platforms.
Post-training samples to improve performance
In addition to the base model, we will release reference post-training checkpoints and training recipes to help developers adapt and optimize Cosmos 3 for their own applications.
Cosmos 3 is an open-world foundational model platform. The model is post-trained using high-quality domain-specific data, so its accuracy continues to improve for specialized applications. Cosmos serves as a starting point for building domain-adapted models of the world using open frameworks. Post-training allows developers to improve model performance while maintaining output quality. To demonstrate this workflow, we are also releasing a Cosmos 3 Super 4-Step Distillation checkpoint and post-training script. This provides a faster reference implementation of the 64B model to help developers adapt Cosmos to their own domains and achieve downstream performance improvements.
Cosmos 3 Super 4 Steps (Text-to-Image and Image-to-Video): These are extracted checkpoints and scripts for distribution matching that reduce diffusion from 35-50 denoising steps to just 4 steps, delivering up to 25x faster inference while maintaining image and video quality and fidelity.
These examples serve as reference implementations that developers can build upon. Open training scripts allow you to post-train Cosmos 3 Edge for custom robot policies or extract Cosmos 3 Super to generate images and videos faster for your applications.
Try Cosmos 3 Edge
Download the model: https://huggingface.co/nvidia/Cosmos3-Edge
Cosmos 3 Edge connects understanding, prediction, simulation, and action through a shared world representation. Developers can deploy Cosmos 3 Edge closer to the sensors on the device. Models can be used as reasoners or action generators.
Customize and specialize your models using open Cosmos Frameworks.
Check out Cosmos 3: https://huggingface.co/collections/nvidia/cosmos3
We plan to continue to advance Cosmos 3 for Physical AI to generate interactive worlds, simulate driving scenarios, and improve robotics policies.
We’re also investing in faster inference and more efficient post-training across a wider range of hardware, reducing the time and compute required to build downstream models.
This includes Cosmos 3 checkpoint optimization using open inference and optimization frameworks such as vLLM, with more optimizations and developer tools coming soon.
