Reinforcement Learning: An Introduction (2nd Edition)
FreeRL fundamentals (Sutton & Barto).
FreeFree tier
About Reinforcement Learning: An Introduction (2nd Edition)
Reinforcement Learning: An Introduction (2nd Edition) is the definitive textbook on reinforcement learning, written by pioneers Richard S. Sutton and Andrew G. Barto. It provides a comprehensive and accessible introduction to the theory and algorithms of reinforcement learning, including Markov decision processes, dynamic programming, Monte Carlo methods, temporal-difference learning, n-step bootstrapping, function approximation, and policy gradient methods. The book is widely used in university courses and as a reference for researchers and practitioners. The second edition is freely available online as a PDF.
Key Features
Markov decision processes and Bellman equations
Dynamic programming (policy and value iteration)
Monte Carlo methods for prediction and control
Temporal-difference learning (SARSA, Q-learning)
n-step bootstrapping and eligibility traces
Function approximation (linear, neural networks)
Policy gradient methods (REINFORCE, actor-critic)
Planning and learning with models (Dyna, Monte Carlo tree search)
Exploration and exploitation (ε-greedy, UCB, Thompson sampling)
Off-policy learning and importance sampling
Pros & Cons
Pros
- Authored by pioneers Richard S. Sutton and Andrew G. Barto
- Comprehensive coverage of classical and modern RL algorithms
- Freely available online as a PDF (second edition)
- Widely used and cited in academia and industry
- Clear explanations with illustrative examples and diagrams
- Includes exercises and problem sets for deeper understanding
Cons
- Dense mathematical content may be challenging for beginners without prior ML background
- Some topics (e.g., deep RL) are covered only briefly or in later chapters
- Lacks interactive code examples or programming exercises
- Physical book is expensive, though PDF is free
Best For
Learning the fundamentals of reinforcement learning from scratchReference textbook for RL researchers and practitionersTeaching material for university courses on reinforcement learningSelf-study for machine learning engineers and data scientists
FAQ
What is Reinforcement Learning: An Introduction about?
It is a comprehensive textbook covering the theory and algorithms of reinforcement learning, from basic Markov decision processes to advanced topics like policy gradients and function approximation.
Is the book free?
Yes, the second edition PDF is freely available from the authors' website (and linked from Stanford's course page).
Who are the authors?
Richard S. Sutton and Andrew G. Barto, both leading researchers in reinforcement learning.
What programming language or code is included?
The book does not include code examples; it focuses on theory and algorithms. However, many companion code repositories exist online.
Is this book suitable for beginners?
It is suitable for readers with a basic understanding of probability, statistics, and machine learning. The early chapters are gentle, but later chapters require more mathematical maturity.