\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

DL01 · Neural Network Foundations

A free, illustrated L2 course of 10 modules in the Deep Learning track. Modules include: From linear models to neurons; Multilayer perceptrons and universal approximation; Activation functions.

DL01 L2 · Intermediate 10 modules · about 40 lessons Planned · Ring 2

Part of the Deep Learning track. Ships in Ring 2 (Spine) of the roadmap.

Modules

1. From linear models to neurons
2. Multilayer perceptrons and universal approximation
3. Activation functions
4. Loss functions and maximum likelihood
5. Backpropagation by hand
6. Build an autodiff engine from scratch
7. PyTorch fundamentals
8. Anatomy of a training loop
9. Datasets, loaders and batching
10. First project: an image classifier

Signature illustrations

The pictures a learner should still see in their head a year later:

Neuron and layer diagrams, depth as folding of input space, backpropagation on a computational graph, slices of a loss surface, counting linear regions.

Status

This course is planned, not written yet. It ships with Ring 2 (Spine).

Every lesson will follow the same twelve-part blueprint with at least three purpose-made figures. Watch the repository to hear when modules ship, or help write and review it.