NVIDIA Expands AI Toolkit with New Cosmos Models for Robotics and Autonomous Systems

NVIDIA has unveiled a new suite of infrastructure and “world models” under its Cosmos platform, designed to accelerate the development of physical AI applications.
This strategic move, announced at the SIGGRAPH conference, signifies a major push to extend generative AI beyond digital realms and into real-world autonomous systems like robotics and self-driving vehicles. The new Cosmos platform introduces a family of pretrained world foundation models (WFMs), including Cosmos Predict, Cosmos Transfer, and Cosmos Reason, each serving a distinct purpose in the AI development pipeline. Cosmos Predict, for example, can generate predictive videos from various inputs, while Cosmos Transfer helps in creating photorealistic data from 3D simulations.
The most notable addition is Cosmos Reason, a customizable vision-language model designed to help AI agents understand and reason about the physical world, a crucial step for real-time video analytics and robot decision-making.
By making these models open and accessible to developers, NVIDIA is not only solidifying its position as a leader in accelerated computing but also providing the foundational tools necessary to train physical AI systems more safely and efficiently in a virtual environment before real-world deployment. This approach aims to dramatically reduce the time, cost, and risks associated with real-world testing.