World Model: Definition and Examples

A world model refers to the internal representation that an artificial intelligence agent builds of its environment. This representation encodes the rules, causal relationships, and dynamics of the real world, allowing the system to 'mentally simulate' scenarios before acting. It is the computational equivalent of our ability to imagine what would happen if we performed a particular action.

The concept has its roots in cognitive science and robotics, where researchers have long sought to equip machines with an internal understanding of their environment. In the context of large language models (LLMs), the debate rages: do these models develop true world models through training on vast text corpora, or do they merely reproduce sophisticated statistical patterns? Recent work suggests that the most advanced LLMs do build structured internal representations of the world, even if they remain imperfect.

In practice, a good world model allows an AI to generalize beyond its training data, reason about novel situations, and anticipate the consequences of its responses. For example, when an LLM solves a physics problem, it mobilizes a form of world model incorporating the laws of mechanics. World models are also at the heart of 'model-based' AI in reinforcement learning, where the agent learns a model of its environment to plan its actions optimally.

The challenge of world models is central to the future of AI: a system with a robust and faithful world model would be capable of causal reasoning, long-term planning, and adaptation to new contexts — capabilities considered essential on the road to artificial general intelligence (AGI). Yann LeCun, among others, places world models at the core of his architectural vision for the next generation of AI systems.