What is a world model, exactly?

DIRECT ANSWER · VERIFIED SOURCES · 2026-07-22

What is a world model, and why is every video generator not automatically one?

A world model is a learned internal model of environmental state and change: it uses observation history to form a state representation, predicts futures under actions or conditions, and lets an agent evaluate behavior through those predictions. A video generator only needs visually plausible frames. Without stable state, action-conditioned consequences, long-horizon consistency, or demonstrated planning utility, world-like imagery alone does not establish a useful world model.

Key takeaways

• The core interface is state, observation, action, and transition—not merely text-to-video.
• Multi-step prediction, counterfactual control, and planning provide stronger evidence than visual quality.
• Pixel generation, latent prediction, and persistent 3D can all be implementation routes, but labels are not capability evidence.

Boundary and caveat

Research uses several overlapping definitions, and some work calls large video predictors world models. This course uses a functional test: the model must represent environmental change and be independently evaluated for prediction, control, or planning.

Primary sources

• Ha & Schmidhuber · World Models
• LeCun · A Path Towards Autonomous Machine Intelligence
• Hafner et al. · DreamerV3

state

state is an explicit variable in the world-model loop: state produces observations, actions alter subsequent state, and the model supplies conditional counterfactuals for planning. Label whether each value comes from the environment, a sensor, the policy, or the model rather than treating a prediction as ground truth.

observation

observation is an explicit variable in the world-model loop: state produces observations, actions alter subsequent state, and the model supplies conditional counterfactuals for planning. Label whether each value comes from the environment, a sensor, the policy, or the model rather than treating a prediction as ground truth.

action

action is an explicit variable in the world-model loop: state produces observations, actions alter subsequent state, and the model supplies conditional counterfactuals for planning. Label whether each value comes from the environment, a sensor, the policy, or the model rather than treating a prediction as ground truth.

transition

transition is an explicit variable in the world-model loop: state produces observations, actions alter subsequent state, and the model supplies conditional counterfactuals for planning. Label whether each value comes from the environment, a sensor, the policy, or the model rather than treating a prediction as ground truth.

prediction

prediction is an explicit variable in the world-model loop: state produces observations, actions alter subsequent state, and the model supplies conditional counterfactuals for planning. Label whether each value comes from the environment, a sensor, the policy, or the model rather than treating a prediction as ground truth.