The Rise of World Models: An Alternative to Large Language Models

Large language models (LLMs) are encountering limitations in domains requiring an understanding of the physical world—from robotics and autonomous driving to manufacturing. This constraint is driving investor interest toward world models, with AMI Labs raising $1.03 billion shortly after World Labs secured $1 billion. While LLMs excel at processing abstract knowledge through next-token prediction, they lack grounding in physical causality, hindering their ability to reliably predict the consequences of real-world actions.

The Limitations of Large Language Models

AI researchers and thought leaders are increasingly vocal about the limitations of LLMs as the industry attempts to move AI beyond web browsers and into physical spaces. Turing Award recipient Richard Sutton has warned that LLMs merely mimic human speech rather than modeling the world, limiting their capacity to learn from experience and adapt to changing conditions. This can lead to brittle behavior, where even small changes to inputs cause models to fail.

Google DeepMind CEO Demis Hassabis has echoed this sentiment, describing today’s AI models as suffering from “jagged intelligence.” They can solve complex mathematical problems but struggle with basic physics due to a lack of understanding of real-world dynamics. To address this, researchers are shifting their focus to building world models that act as internal simulators, allowing AI systems to safely test hypotheses before taking physical action.

What are World Models?

“World models” is an umbrella term encompassing several distinct architectural approaches, each with different tradeoffs. Three main approaches are emerging:

Joint Embedding Predictive Architecture (JEPA)

The first approach focuses on learning latent representations instead of predicting the dynamics of the world at the pixel level. AMI Labs champions this method, heavily based on the Joint Embedding Predictive Architecture (JEPA). JEPA models aim to mimic human understanding by focusing on core rules of interaction rather than memorizing every detail. For example, when observing a car, we track its trajectory and speed, not the exact reflection of light on every leaf.

This architecture is computationally and memory efficient, requiring fewer training examples and offering lower latency, making it suitable for applications like robotics, self-driving cars, and high-stakes enterprise workflows. AMI Labs is partnering with healthcare company Nabla to use this architecture to simulate operational complexity and reduce cognitive load in healthcare settings. Yann LeCun has stated that JEPA-based world models are designed to be “controllable,” allowing them to achieve specified goals.

Gaussian Splats

A second approach uses generative models to build complete spatial environments. Companies like World Labs employ this method, taking an initial prompt (image or text) and creating a 3D Gaussian splat—a technique representing 3D scenes using mathematical particles defining geometry, and lighting. These representations can be imported into physics and 3D engines like Unreal Engine for interactive navigation.

This drastically reduces the time and cost of creating complex 3D environments, addressing the spatial intelligence gap identified by Fei-Fei Li, founder of World Labs, who noted that LLMs are “wordsmiths in the dark.” While not designed for real-time execution, this approach has potential for spatial computing, entertainment, industrial design, and robotics training. Autodesk’s backing of World Labs demonstrates its enterprise value.

End-to-End Generation

The third approach uses an end-to-end generative model to continuously generate scenes, dynamics, and reactions on the fly. Models like DeepMind’s Genie 3 and Nvidia’s Cosmos fall into this category, providing a simple interface for generating interactive experiences and synthetic data. DeepMind demonstrated Genie 3’s ability to maintain object permanence and consistent physics at 24 frames per second without a separate memory module.

This translates into powerful synthetic data factories, allowing autonomous vehicle and robotics developers to synthesize rare, dangerous scenarios without physical testing. Waymo built its world model on top of Genie 3 for self-driving car training. The downside is the high computational cost of continuously rendering physics and pixels, but the investment is deemed necessary for achieving a deep understanding of physical causality.

The Future of AI: Hybrid Architectures

LLMs will likely continue to serve as the reasoning and communication interface, while world models position themselves as foundational infrastructure for physical and spatial data pipelines. As these models mature, hybrid architectures are emerging, drawing on the strengths of each approach. Cybersecurity startup DeepTempo recently developed LogLM, integrating elements from LLMs and JEPA to detect anomalies and cyber threats from security logs.