NVIDIA Introduces Cosmos World Foundation to Boost Physical AI Innovation

CES Las Vegas

NVIDIA introduces Cosmos: A Game-Changer for Physical AI

Today, nVidia has rolled out something that could reshape the landscape of physical AI: meet NVIDIA Cosmos. This new platform is packed with generative world foundation models, advanced tokenizers, guardrails, and a turbocharged video processing pipeline. It’s designed to supercharge the development of physical AI systems, including autonomous vehicles (AVs) and robots.

Why is this such a big deal? Well, building physical AI models isn’t just a walk in the park. It’s a costly endeavor that demands vast amounts of real-world data and rigorous testing. Enter the Cosmos world foundation models, or WFMs. These models are here to simplify the process for developers, allowing them to generate

massive amounts of photorealistic

, physics-based synthetic data. This means that training and evaluating existing models just got a whole lot easier. Plus, developers can tweak these WFMs to create custom models that fit their unique needs. And the best part? These models will be available under an open model license, making it easier for the robotics and AV communities to get involved.

If you’re itching to try it out, you can check out the first models on the NVIDIA API catalog or grab the entire suite of models and fine-tuning framework from the NVIDIA NGC catalog or Hugging Face. Big names in robotics and automotive, including Uber and companies like 1X, Agile Robots, and XPENG, are already on board.

A Democratizing Force in Physical AI

Jensen Huang, NVIDIA’s founder and CEO, put it perfectly: “The ChatGPT moment for robotics is coming.” Just like large language models have revolutionized text-based AI, Cosmos WFMs are set to propel the development of robots and AVs. But here’s the catch—

not every developer has the resources or expertise

to train their own models. That’s where Cosmos steps in, aiming to democratize physical AI and make general robotics accessible to all developers.

Unlocking New Possibilities with Open World Foundation Models

So, what exactly can developers do with these open models? The possibilities are pretty exciting. Cosmos WFMs are tailored for physical AI research and development, capable of generating physics-based videos from various inputs, including text, images, and even robot sensor data. Imagine creating simulated environments that mimic real-world conditions—like navigating a busy warehouse or driving on a snowy road.

During his keynote at CES, Huang showcased how developers can leverage Cosmos models for tasks like video search and understanding, which helps pinpoint specific training scenarios from video data. They can also generate photorealistic synthetic data using the NVIDIA Omniverse platform, or even evaluate and improve custom models through reinforcement learning.

Advanced Tools for a New Era of AI Development

Creating physical AI models typically requires enormous amounts of video data and a staggering amount of compute time—think petabytes of data and tens of thousands of hours. But with Cosmos, NVIDIA is cutting down those costs significantly. For instance, the NVIDIA AI and CUDA-accelerated data processing pipeline allows developers to process and curate a whopping 20 million hours of video in just 14 days, rather than the three years it would take with traditional methods.

Plus, the Cosmos Tokenizer is a game-changer, converting images and videos into tokens with 8x more compression and 12x faster processing than existing options. Coupled with the NVIDIA NeMo framework, developers can train and customize models more efficiently than ever before.

Industry Leaders Are Already Onboard

The excitement around Cosmos isn’t just hype; industry pioneers are already diving in. For example, 1X has launched the 1X World Model Challenge dataset using the Cosmos Tokenizer, while XPENG is leveraging Cosmos to speed up the development of its humanoid robot. Even companies like Waabi and Wayve are exploring how Cosmos can enhance their AV software development and safety validation.

Pras Velagapudi, CTO at Agility, summed it up well: “Data scarcity and variability are key challenges to successful learning in robot environments.” With Cosmos, developers can generate photorealistic scenarios that help train models without the hefty price tag of real-world data capture.

A Commitment to Safe and Responsible AI

NVIDIA has also made it clear that Cosmos is built with

trustworthy AI principles

in mind. This means prioritizing privacy, safety, security, and transparency, all while reducing bias. The platform includes guardrails to mitigate harmful content and features tools to enhance text prompts for accuracy. Plus, videos generated with Cosmos come with invisible watermarks to help combat misinformation.

NVIDIA is encouraging developers to adopt these trustworthy practices and improve their own applications with robust guardrails and watermarking solutions.

How to Get Started with Cosmos

Excited to dive into the world of Cosmos? The WFMs are now available under NVIDIA’s open model license on Hugging Face and the NVIDIA NGC catalog. Developers can also utilize NVIDIA NeMo Curator for accelerated video processing and customize their own world models. And if you’re looking for a hassle-free deployment, NVIDIA DGX Cloud offers a quick and easy way to get these models up and running, complete with enterprise support through the NVIDIA AI Enterprise software platform.

And that’s not all! NVIDIA has also announced new large language models, including NVIDIA Llama Nemotron and NVIDIA Cosmos Nemotron vision language models, designed for enterprise AI applications across various industries like healthcare and finance.

With NVIDIA Cosmos, the future of physical AI is brighter than ever. Are you ready to be part of this revolution?

Background Information

---

About nVidia:

NVIDIA has firmly established itself as a leader in the realm of client computing, continuously pushing the boundaries of innovation in graphics and AI technologies. With a deep commitment to enhancing user experiences, NVIDIA's client computing business focuses on delivering solutions that power everything from gaming and creative workloads to enterprise applications. for its GeForce graphics cards, the company has redefined high-performance gaming, setting industry standards for realistic visuals, fluid frame rates, and immersive experiences. Complementing its gaming expertise, NVIDIA's Quadro and NVIDIA RTX graphics cards cater to professionals in design, content creation, and scientific fields, enabling real-time ray tracing and AI-driven workflows that elevate productivity and creativity to unprecedented heights. By seamlessly integrating graphics, AI, and software, NVIDIA continues to shape the landscape of client computing, fostering innovation and immersive interactions in a rapidly evolving digital world.

  

Latest Articles about nVidia

Event Info

---

About CES:

CES, the Consumer Electronics Show, is an annual event held in Las Vegas, Nevada, organized by the Consumer Technology Association (CTA). With a history dating back to 1967, it has become the world's premier platform for unveiling and exploring the latest innovations in consumer electronics and technology. Drawing exhibitors ranging from industry titans to startups across diverse sectors, including automotive, health and wellness, robotics, gaming, and artificial intelligence, CES transforms Las Vegas into a global tech hub, offering a glimpse into the future of technology through a wide array of showcases, from startup-focused Eureka Park to cutting-edge automotive and health tech exhibitions.

  

Latest Articles about CES