LESSON wm.1.3 · Derivation · 140 min
MuZero: learn only the dynamics planning needs
MuZero: learn only the dynamics planning needs. Learn representation, dynamics, prediction, MCTS, value, and policy and complete: Implement simplified model-based planning in a toy
Learning objectives
- Explain what problem “MuZero: learn only the dynamics planning needs” solves without hiding behind terminology.
- Trace the variables and causal links across representation, dynamics, prediction.
- Complete “Implement simplified model-based planning in a toy game” and judge the result with evidence rather than intuition.
Core concepts
representation
representation belongs to MuZero’s task-relevant latent system: representation encodes history, dynamics predicts the next latent state and reward under an action, prediction emits policy and value, and MCTS searches with those quantities. The state need not reconstruct pixels and should not be read as literal physics.
dynamics
dynamics belongs to MuZero’s task-relevant latent system: representation encodes history, dynamics predicts the next latent state and reward under an action, prediction emits policy and value, and MCTS searches with those quantities. The state need not reconstruct pixels and should not be read as literal physics.
prediction
prediction belongs to MuZero’s task-relevant latent system: representation encodes history, dynamics predicts the next latent state and reward under an action, prediction emits policy and value, and MCTS searches with those quantities. The state need not reconstruct pixels and should not be read as literal physics.
MCTS
MCTS belongs to MuZero’s task-relevant latent system: representation encodes history, dynamics predicts the next latent state and reward under an action, prediction emits policy and value, and MCTS searches with those quantities. The state need not reconstruct pixels and should not be read as literal physics.
value
value belongs to MuZero’s task-relevant latent system: representation encodes history, dynamics predicts the next latent state and reward under an action, prediction emits policy and value, and MCTS searches with those quantities. The state need not reconstruct pixels and should not be read as literal physics.
Build and verify
Implement simplified model-based planning in a toy game
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.