\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

ML02 · Supervised Learning: Trees, Ensembles and Kernels

A free, illustrated L2 course of 11 modules in the Machine Learning track. Modules include: Decision trees; Bagging and random forests; AdaBoost.

ML02 L2 · Intermediate 11 modules · about 44 lessons Planned · Ring 2

Part of the Machine Learning track. Ships in Ring 2 (Spine) of the roadmap.

Modules

1. Decision trees
2. Bagging and random forests
3. AdaBoost
4. Gradient boosting
5. XGBoost, LightGBM and CatBoost
6. Naive Bayes and generative classifiers
7. Maximum margin and support vector machines
8. The kernel trick
9. Discriminant analysis
10. Multi-class, multi-label and ordinal problems
11. Choosing a model for tabular data

Signature illustrations

The pictures a learner should still see in their head a year later:

A tree growing split by split, boosting stages fitting residuals, margin and support vectors, the kernel lift from two to three dimensions, ensemble vote grids.

Status

This course is planned, not written yet. It ships with Ring 2 (Spine).

Every lesson will follow the same twelve-part blueprint with at least three purpose-made figures. Watch the repository to hear when modules ship, or help write and review it.