You have a favorite restaurant. Going back is exploitation: you know the meal will be good. Trying the new place down the street is exploration: it might be worse, or it might be your new favorite, and you only find out by going. Always exploiting means you never discover anything better. Always exploring means you rarely enjoy what you’ve found. Every system that learns from trial and error has to balance the two.
In reinforcement learning, the agent faces this on every step. Sutton and Barto’s textbook introduces it with the multi-armed bandit, a row of slot machines with unknown payouts: pull the machine that has paid best so far, or pull others to learn their odds? Simple strategies include choosing a random action a small fraction of the time, or favoring options whose value is still uncertain. Algorithms such as Q-learning need some exploration to learn anything beyond their first guesses.
The same trade-off appears when language models are trained with policy optimization. Sampling several varied answers to each prompt lets training find better ones; if the model only ever produces its current best guess, it has nothing new to learn from. Outside model training, the idea shows up in product experiments, ad allocation and recommendation feeds, where a business must decide how much traffic to spend learning about new options instead of earning from the proven one.
Sources
- Sutton and Barto: Reinforcement Learning: An Introduction (2nd edition, 2018) — The standard reinforcement learning textbook, free online from the authors, which introduces the exploration-exploitation dilemma through multi-armed bandits in chapter 2; checked October 24, 2026.
- OpenAI Spinning Up: Key Concepts in RL — OpenAI's introductory reinforcement learning documentation covering policies, rewards and the agent-environment loop; checked October 24, 2026.