World model
In model-based reinforcement learning, a world model predicts how an environment changes, often from observations and actions. A learned world model is fitted from experience and can support planning or training in imagined trajectories.
A robot team could fit a model to logs of camera observations and steering actions. Given a candidate turn, the model predicts the next observation or a compact representation of it. Rolling several actions forward lets a planner compare possible routes before trying them in the physical environment.
Ha and Schmidhuber’s World Models paper demonstrates learning compact visual and temporal representations and training a controller in a learned environment. A hand-written simulator can also support model-based planning, but its dynamics are specified rather than learned from data.
Terminology varies: a product called a world model may predict video without exposing an action-controlled planning interface. Ask what it predicts and how actions enter the model. Predictions can accumulate errors over a rollout, so an attractive imagined route is not evidence that a robot can execute it safely. Check plans against the real task.
Sources
- Ha and Schmidhuber: World Models — Learns compact spatial and temporal representations and trains policies inside a learned world model.
Go deeper
- Hafner et al.: Mastering Diverse Domains through World Models paper
Explore Dreamer’s use of learned dynamics and imagined trajectories for policy learning.