\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

RL01 · RL Foundations

A free, illustrated L2 course of 9 modules in the Reinforcement Learning track. Modules include: The agent and environment loop; Multi-armed bandits; Exploration: epsilon-greedy, UCB and Thompson sampling.

RL01 L2 · Intermediate 9 modules · about 36 lessons 1 of 9 modules live

Part of the Reinforcement Learning track. Ships in Ring 2 (Spine) of the roadmap.

Modules

1. The agent and environment loop
2. Multi-armed bandits
3. Exploration: epsilon-greedy, UCB and Thompson sampling
4. Markov decision processes
5. Returns, discounting and value functions
6. Bellman equations Live · 4 lessons
7. Policy evaluation
8. Policy iteration and value iteration
9. Contextual bandits

Signature illustrations

The pictures a learner should still see in their head a year later:

The agent-environment loop, bandit regret curves, gridworld value heatmaps with policy arrows, backup diagrams.

Status

1 of 9 modules are written and live. The rest follow in Ring 2.

Every lesson will follow the same twelve-part blueprint with at least three purpose-made figures. Watch the repository to hear when modules ship, or help write and review it.