Interview

World Models: An intuitive introduction

Source attribution: Original source

Interview details

Publication date: Aug 14, 2026
Duration: 1h 14m

Speakers

Ankit GuptaGeneral Partner, Y Combinator Francois ChaubardPhD student, Stanford University

Viewpoints in this interview

Intelligence per sample is a major unsolved problem in AI.

The speaker defines the core challenge as measuring how much smarter a model becomes with each additional data point, noting that current models are very inefficient at learning from small datasets compared to humans.

Francois Chaubard Sample Efficiency Source at 1:01

Perfect sample efficiency is achievable with a perfect world model.

The speaker explains that zero-sample learning is theoretically possible if one has a flawless predictive model of the environment, using Newtonian physics and model predictive control as an example of applying known laws to plan actions without real-world trials.

Francois Chaubard World Models Source at 3:00

World models enable pre-programmed planning without environmental sampling.

Using the example of NASA's asteroid interception, the speaker illustrates that a perfect world model allows for pre-calculated trajectories that don't require adaptive sampling from the environment during execution.

Ankit Gupta World Models Source at 3:37

Control problems are solved using transition functions and policies.

The speaker defines the core elements of a control problem: a state, an action (control input), a world model (state transition function), and a policy (what action to take given a state).

Francois Chaubard Reinforcement Learning Source at 8:26

Non-differentiable stochastic systems require reinforcement learning methods.

When an environment becomes stochastic and non-differentiable (e.g., due to an unpredictable adversary), direct optimization fails, necessitating reinforcement learning techniques like value iteration or actor-critic.

Francois Chaubard Reinforcement Learning Source at 12:41

World models help estimate value functions and joint state-action distributions.

The speaker clarifies that in standard reinforcement learning, machine learning models are used to estimate value functions, and world models provide a framework for jointly modeling state transitions and policies to create more intelligent behavior.

Ankit Gupta Reinforcement Learning Source at 16:19

World models and policies are combined into a joint probability distribution.

The speaker describes the desired joint distribution for intelligent agents, which factorizes into a policy and a world model, and notes these are often learned separately before being combined.

Francois Chaubard World Models Source at 16:46

Action conditioning is added to world models to enable interaction.

Building on passive video prediction, action conditioning is injected into world models to allow them to simulate the consequences of actions, a technique central to the Dreamer paper series.

Francois Chaubard World Models Source at 17:36

Jointly trained world-action models are faster for test-time planning.

The speaker argues that jointly training a single model to output both actions and next states is more efficient and faster for planning than using separate, sequential models.

Francois Chaubard World Models Source at 18:25

AlphaGo uses Monte Carlo Tree Search for expensive test-time planning.

The speaker details how AlphaGo uses MCTS, performing many model invocations to build a search tree that balances exploration and exploitation to select optimal moves in games with small action spaces.

Francois Chaubard Monte Carlo Tree Search Source at 26:40

MCTS scaling is limited by large action spaces and real-time requirements.

The speaker explains that the MCTS approach used in AlphaGo does not scale to domains like self-driving or robotics because of large action spaces, non-deterministic environments, and the need for real-time decisions.

Francois Chaubard Monte Carlo Tree Search Source at 33:14

Self-driving car state and action spaces are effectively infinite.

The speaker describes the state space of a self-driving car as massive (including surroundings, vehicle state, weather) and calculates that even a simplified action space (steering, brake, gas) is vastly larger than in games like Go.

Ankit Gupta Self-Driving Source at 34:14

Self-driving requires modeling the impact of actions on other agents.

The speaker notes that while Newtonian physics can model car dynamics, the non-differentiable element is predicting how other road users will react to one's own actions, which is essential for safe navigation.

Francois Chaubard Self-Driving Source at 36:59

Tesla's fleet provides a unique dataset of driver actions.

The speaker highlights a key competitive advantage for Tesla: its fleet of consumer vehicles collects video data paired with the driver's actual actions (steering, braking), which most other companies lack.

Francois Chaubard Self-Driving Source at 39:31

Model-free reinforcement learning predicts actions directly from states.

The speaker defines model-free reinforcement learning as a method that learns a policy (action from state) without an explicit model of the environment's dynamics, analogous to behavior cloning or next-token prediction.

Francois Chaubard Reinforcement Learning Source at 41:28

Model-based reinforcement learning uses a world model for planning.

The speaker contrasts model-free methods with model-based RL, where an explicit world model (the transition function) is used to plan actions, enabling stronger policies but requiring more computation for test-time inference.

Francois Chaubard Reinforcement Learning Source at 42:21

Dreamer trains policies on synthetic data from a world model.

The speaker describes the Dreamer series as pioneering work that trains a world model from environment data and then generates synthetic rollouts to train a policy, achieving results like mining diamonds in Minecraft using only synthetic data.

Francois Chaubard Dreamer Architecture Source at 48:55

Video generation models can be fine-tuned into actionable world models.

The speaker explains that state-of-the-art video diffusion models (like Sora) can be fine-tuned with a small amount of action-conditioned data to create effective world models for robotics and self-driving.

Francois Chaubard Dreamer Architecture Source at 51:29

Dreamer V4 for robotics uses pre-trained diffusion models and tele-operation data.

The speaker cites a paper applying the Dreamer concept to robotics, which starts with an open-source video diffusion model and fine-tunes it with about 500 hours of tele-operation data to achieve cross-embodiment tasks.

Ankit Gupta Dreamer Architecture Source at 53:14

JEPA operates in latent space to improve sample efficiency.

The speaker introduces Joint Embedding Predictive Architecture (JEPA) as a method that compresses high-dimensional state spaces (like images) into latent vectors and performs world modeling and predictions in that efficient latent space.

Francois Chaubard Joint Embedding Predictive Architecture Source at 58:49