\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

P02.9 · Reproducible notebooks

A free, illustrated module of 4 lessons and one interactive in P02 Scientific Python: NumPy, pandas and Plotting: Kernel, cells and hidden state; Randomness you can repeat; Data, paths and versions you can find again; From scratchpad to report.

P02.9 L1 · Core 4 lessons · about 100 min Live · in review

Part of P02 Scientific Python: NumPy, pandas and Plotting in the Programming and Data track.

A notebook shows its cells in page order, but the kernel behind it remembers them in the order they were run. These four lessons build a notebook runner from a dictionary and exec, read a notebook file with the json module, make random numbers repeat by fixing a seed, pin down the data, paths and library versions a result depends on, and turn a messy exploration of the weather readings into a report that reruns from top to bottom with no one watching.

What you need first. Functions and scope (P01.4), modules and pinned packages (P01.5), files and JSON (P01.7) and first tests with assert (P01.8). The last lessons use pandas tables, group-by and a Matplotlib chart (P02.4 to P02.8).

Lessons

1. Kernel, cells and hidden state P02.9.1 25 min

   A notebook shows its cells in page order, but the kernel behind it remembers them in the order they were run. Build a kernel from a dictionary and exec, run three cells three ways, read a notebook file with the json module, and learn the one test that catches hidden state.

2. Randomness you can repeat P02.9.2 25 min

   A computer’s random numbers come from a rule and a starting state, the seed. Build a tiny generator by hand, then use NumPy’s default_rng, see why every call moves its state, and seed pandas and scikit-learn so that a shuffle or a split comes out the same every time.

3. Data, paths and versions you can find again P02.9.3 25 min

   A notebook’s result depends on more than its code: the data file it reads, the path it reads it from, the library versions it runs on and the machine’s last digits. Build paths from the project folder, fingerprint data with SHA-256, record and pin versions, and compare floats with np.allclose.

4. From scratchpad to report P02.9.4 25 min

   Turn a messy exploration of the weather readings into a notebook that reads top to bottom: question, imports and parameters, load, clean, summarise, plot, conclusion. Move repeated cells into tested functions and a module, then run the report with no one watching.

The interactive

P02.9-X1 · Run-order sandbox. Three cells, x = 10, x = x + 1 and y = x * 2, each with a run button and an execution counter, beside the kernel’s variables. Run them in any order or several times, then Restart and Run All: the read-out says whether what was on screen matches a clean run. It lives in Kernel, cells and hidden state.