\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

P02.7 · Dates, text and categories

A free, illustrated module of 4 lessons and one interactive in P02 Scientific Python: NumPy, pandas and Plotting: Timestamps: dates as numbers; Resampling and rolling windows; Text columns and the str accessor; Categories and encoding them as numbers.

P02.7 L1 · Core 4 lessons · about 100 min Live · in review

Part of P02 Scientific Python: NumPy, pandas and Plotting in the Programming and Data track.

Real tables hold more than numbers. These four lessons turn dates into counts of time since 1970 that can be ordered, subtracted and sliced, summarise a fortnight of temperatures by week and with sliding windows, clean hand-typed text labels with the vectorised str methods and a first regular expression, and store a column of labels as categories whose integer codes become one-hot features for a model.

What you need first. Strings and their methods (P01.7), NumPy arrays and broadcasting (P02.1, P02.2), Series and DataFrames with loc and label alignment (P02.4), reading and cleaning a file (P02.5) and group-by as split-apply-combine (P02.6).

Lessons

1. Timestamps: dates as numbers P02.7.1 25 min

   A timestamp is a count of time units since 1 January 1970. Count the days to 3 March 2025 by hand, then parse a text column with pd.to_datetime, read its parts with .dt, subtract dates, slice by date, and learn why 03/04/2025 needs its layout stated.

2. Resampling and rolling windows P02.7.2 25 min

   Use time as a group key. Compute weekly means of a fortnight of temperatures with a dict, then with resample(‘W’); slide a 3-day window by hand, then with rolling(3); build lag features with shift and diff, and see why a centred window leaks the future.

3. Text columns and the str accessor P02.7.3 25 min

   Clean hand-typed labels such as ’ Coast’ and ‘HILL’ with a comprehension, then with the vectorised .str methods that pass missing values through. Split codes into columns, and pull the number out of ‘21.5 C’ with a first regular expression.

4. Categories and encoding them as numbers P02.7.4 25 min

   A categorical column stores each label once and a small integer code per row. Build the look-up table and the codes for the sky column by hand, declare an order for the days, and turn codes into one-hot columns with broadcasting and pd.get_dummies.

The interactive

P02.7-X1 · Resample and roll. The 14 daily valley temperatures as dots on a time axis. Choose weekly bins with a mean, min or max, or a rolling window of 1 to 7 days, trailing or centred, and step a selected day along the fortnight to see which dots feed its value and how many values are missing. It lives in Resampling and rolling windows.