One often-repeated phrase at this year’s Nvidia GTC was «Physical AI,» the kind of AI that allows robots to operate effectively in the physical world.
To get there, Nvidia offers its Isaac GR00T N1, an open-source foundation model that is pre-trained with some of the basics and broken down into a fast system for taking action and a slow system for planning action.
For teaching robots new tasks, lots of varied data is needed. Tele-operation and video demonstrations are important but usually not enough. Synthetic data generated from training in simulation helps fill the gap, and this is where Nvidia’s Omniverse with Cosmos comes in.
«Omniverse is a platform that brings together data between different sources and allows you to build a digital twin that’s physically accurate,» says Akhil Docca, Senior Project Marketing Manager for Omniverse. «Cosmos is important because it allows not only the augmentation of data for photorealism, it also allows you to create an exponentially large [amount] of data.»
This was illustrated during Nvidia CEO Jensen Huang’s keynote speech when a digital twin of a robot was shown in candy-like bright colors in Omniverse. The same image was shown cycling through several different iterations with lifelike shading and texture added by Cosmos using simple text prompts.
To make that synthetic data as applicable to the real world as possible, a detailed representation of the laws of physics is required. Nvidia partnered with DeepMind and Disney to develop the open-source physics engine Newton. During his keynote, Huang brought out a remote-controlled BDX droid to celebrate the announcement.
Much of this work on humanoid robots is framed as an answer to looming anxiety about labor shortages. In his keynote, Huang predicted a worldwide shortage of 50 million workers by the end of the decade and envisioned a future where we might pay robots «$50,000 per year» to work.
To get more clarity on Huang’s comments, I contacted Nvidia, which told me the predicted shortage of 50 million workers by the end of the decade was based on a combination of estimates from various sources, industries and locations. The sources Nvidia provided included estimates regarding manufacturing, renewable energy, trucking, nursing, hospitality workers in the United States and Europe, domestic workers in US and Europe, and construction workers in the US and Europe.
A representative from Nvidia also clarified the company’s anticipated pricing model for humanoid robots, saying «It’s more like subscribing to full self-driving services for an autonomous vehicle. The customer buys the car and then pays for the services it wants on top. We believe robots will be the same. Buy the robot and pay for services on top.»
To see these technologies in action, check out the video in this article.
