Scaling Physical AI with OpenUSD Core Specification 1.0
Reading Time: 4 mins
Editor’s note: This post is part of Into the Omniverse, a series focused on how developers, 3D practitioners and enterprises can transform their workflows using the latest advancements in OpenUSD and NVIDIA Omniverse.
Physical AI is moving from research labs into the real world, powering intelligent robots and autonomous vehicles (AVs) – such as robotaxis – that must reliably sense, reason and act amid unpredictable conditions.
To safely scale these systems, developers need workflows that connect real-world data, high-fidelity simulation and robust AI models atop the common foundation provided by the OpenUSD framework.
The recently published OpenUSD Core Specification 1.0 – OpenUSD – aka Worldwide Scene Description – now defines standard data types, file formats and composition behaviors, giving developers predictable, interoperable USD pipelines as they scale autonomous systems.
Powered by OpenUSD, NVIDIA Omniverse libraries combine NVIDIA RTX rendering, physics simulation and efficient runtimes to create digital twins and simulation-ready (SimReady) assets that accurately reflect real-world environments for synthetic data generation and testing.
NVIDIA Cosmos world foundation models can run on top of these simulations to amplify data variation, generating new weather, lighting and terrain conditions from the same scenes so teams can safely cover rare and challenging edge cases.
Learn more by watching the OpenUSD livestream today at 11 a.m. PT or in replay, part of the NVIDIA Omniverse OpenUSD Insiders series:
In addition, advancements in synthetic data generation, multimodal datasets and SimReady workflows are now converging with the NVIDIA Halos framework for AV safety, creating a standards-based path to safer, faster, more cost-effective deployment of next-generation autonomous machines.
“`html
Reducing AV Testing Costs with Simulation and statistical Combination
Published: 2025/12/18 13:22:18
Autonomous vehicle (AV) development requires extensive testing to ensure safety and reliability. Traditionally, this has meant accumulating millions of miles of real-world driving data – a process that is both expensive and time-consuming. Though, recent advancements in simulation and statistical methods are offering a promising path to reduce these costs while maintaining rigorous safety standards. Researchers,with collaborators at Harvard University and Stanford University,recently introduced the Sim2Val framework to statistically combine real-world and simulated test results, reducing AV developers’ need for costly physical mileage while demonstrating how robotaxis and AVs can behave safely across rare and safety-critical scenarios.
The Challenge of AV Testing
Developing safe and reliable autonomous vehicles is a monumental task. AVs must navigate a vast array of scenarios, including common driving situations and rare, potentially risky events (often called “edge cases”). Testing for these edge cases in the real world is impractical due to their infrequent occurrence. waiting for these events to happen naturally would take an unacceptably long time and expose the public to unnecessary risk.This is where simulation becomes crucial.
The Power of Simulation
Simulation allows AV developers to create and test scenarios that are tough or impossible to replicate in the real world. These scenarios can include adverse weather conditions, unexpected pedestrian behavior, and complex traffic patterns. However, simulation isn’t a perfect substitute for real-world testing. Simulations are, by their nature, approximations of reality. discrepancies between the simulated habitat and the real world – known as the “sim-to-real gap” – can lead to AVs performing differently in simulation than they do on the road.
Addressing the Sim-to-Real Gap
The key to effectively using simulation lies in bridging the sim-to-real gap. Traditionally, developers have attempted to improve simulation fidelity, making the virtual world more closely resemble the real one. While this is crucial, it’s also incredibly complex and resource-intensive. The Sim2Val framework takes a different approach.
Introducing Sim2Val: Statistically Combining Real and Simulated Data
Sim2Val doesn’t attempt to eliminate the sim-to-real gap entirely. Instead, it acknowledges its existence and uses statistical methods to account for it. The framework statistically combines results from both real-world testing and simulated testing, weighting each source based on its reliability and relevance. This allows developers to leverage the benefits of both approaches – the cost-effectiveness of simulation and the realism of real-world data.
Here’s how it works:
- Run Simulations: AVs are tested extensively in a variety of simulated scenarios.
- Collect Real-World Data: A smaller amount of real-world driving data is collected.
- statistical Combination: Sim2Val uses statistical techniques to combine the simulation results and the real-world data, adjusting for the known differences between the two environments.
- safety Validation: The combined results provide a more accurate and complete assessment of the AV’s safety performance.
By intelligently combining these data sources, Sim2Val substantially reduces the amount of real-world mileage required to demonstrate safety, lowering development costs and accelerating the deployment of AV technology.
Improving Simulation Assets with NVIDIA Omniverse NuRec Fixer
The quality of simulation relies heavily on the accuracy of the virtual environments and the objects within them. Creating these “SimReady” assets can be challenging, often resulting in artifacts or inaccuracies in neural reconstructions. To address this, NVIDIA has released a new, open-source NVIDIA Omniverse NuRec Fixer, a Cosmos-based model trained on AV data. This tool automatically removes artifacts, producing higher-quality SimReady assets and improving the overall fidelity of the simulation environment.
Looking Ahead
The combination of advanced simulation techniques like Sim2Val and tools for improving simulation asset quality, such as the NVIDIA Omniverse NuRec Fixer, represents a meaningful step forward in AV development.These innovations promise to make the testing process more efficient, cost-effective, and ultimately, safer. As these technologies continue to evolve, we can expect to see even faster progress towards the widespread adoption of autonomous vehicles.