\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

The broadcasting rule

Arrays of different shapes can meet in one expression when their shapes line up from the right. Learn the rule, see a scalar, a row and a column stretched onto a table, write broadcast_shape in plain Python and check it against NumPy.

P02.2.3L1 Core25 min

In review

The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

Before this lesson: P02.2.2 Reductions along an axis

What you will be able to do

State the broadcasting rule: line the shapes up from the right, and on each axis the sizes must be equal or one of them must be 1.
Predict the result shape of an expression between two arrays, or the axis on which it fails.
Draw how a scalar, a row (3,) and a column (4, 1) are stretched onto a (4, 3) table, and how a column and a row make a table together.
Write broadcast_shape in plain Python and check it against np.broadcast_shapes and np.broadcast_to.

The question

temps - temps.mean(axis=0) subtracts each station’s mean from its column, and it just works. temps - temps.mean(axis=1) should subtract each day’s mean from its row, and it stops with a ValueError. Both are a (4, 3) table minus a list of means. What rule does NumPy follow when two arrays of different shapes meet, and why does it accept the first and refuse the second?

From the last lesson

1. What are the shapes of temps.mean(axis=0) and temps.mean(axis=1)?

Show answer

(3,) and (4,): the named axis disappears from the shape (4, 3).

2. What shape does temps.mean(axis=1, keepdims=True) have?

Show answer

(4, 1): the collapsed axis stays, with size 1.

Intuition

In P02.2.1 a number met an array, x * 1.4, and every entry got the same number. Picture the number copied into every cell, so that both sides have the same shape, and then the operation done entry by entry. That picture is called broadcasting, and it goes further than single numbers: a row can be copied down, and a column can be copied across.

P02.2.3-F1

1. Write the two shapes one above the other, lined up from the right. A missing axis counts as 1. 2. Every axis of size 1 is stretched to match. The dashed rows are copies in idea only.

The row of station means has shape (3,). Lined up from the right under (4, 3), its 3 meets the 3 of the stations, a match. On the left it has no axis at all, so it counts as 1, and an axis of size 1 can be stretched to any length: the row is copied down the four days. The two arrays now have the same shape and the subtraction is entry by entry.

The NumPy documentation stresses that the stretching is “only conceptual”. No copies are made in memory: NumPy simply reads the one row again for every day. That is why broadcasting is both short to write and cheap to run.

Build it up

Step 1. A scalar. A single number has shape (), no axes at all. Lined up with (4, 3) it counts as (1, 1), and both axes stretch. So temps - 25 takes 25 from all twelve entries.

Step 2. A row. Shape (3,) against (4, 3): the 3s match on the right, the missing axis counts as 1 and stretches to 4. This is the station means in the anchor figure.

Step 3. A column. Shape (4, 1) against (4, 3): on the right, 1 against 3, so the column is stretched across the three stations; on the left, 4 against 4, a match. This is how the day means of keepdims=True meet the table.

P02.2.3-F2

Left: a scalar is stretched along both axes. Right: a column (4, 1) is stretched along axis 1.

Step 4. A clash. Shape (4,) against (4, 3): lined up from the right, the 4 day means sit over the 3 stations. The sizes differ and neither is 1, so there is nothing to stretch and NumPy raises ValueError: operands could not be broadcast together with shapes (4,3) (4,). That is the failure in the hook: temps.mean(axis=1) has shape (4,), not (4, 1).

P02.2.3-F5

(4, 3) with (4,) fails. Lined up from the right, four day means meet three stations; neither size is 1, so nothing can be stretched.

Step 5. Both arrays stretch. Shapes (4, 1) and (3,) have no common shape to start with, but each has a 1 where the other has a larger size. The column stretches across and the row stretches down, and they meet as a (4, 3) table. The worked example does this with numbers.

The formal statement

The broadcasting rule

To combine arrays with shapes a and b:

Line them up from the right. Compare the last sizes, then the ones before, and so on.
A missing axis counts as 1. The shorter shape is padded with 1s on the left.
Each pair must agree. On every axis the two sizes must be equal, or one of them must be 1. Otherwise NumPy raises a ValueError.
The result takes the other size. On each axis the result has the size that is not 1, or the common size when they are equal. An array whose size there is 1 is stretched to it.

The NumPy user guide states steps 1 and 3 almost word for word: shapes are compared element-wise starting with the trailing, rightmost dimension, and two dimensions are compatible when they are equal or one of them is 1. It adds that missing dimensions are assumed to have size one.

What does “stretched” mean for the numbers? For two arrays that broadcast to a shape (m, n), the entry of the result at row i and column j is built from one entry of each input. On an axis where an input has size 1, that input is read at position 0, whatever the index. For the column col of shape (4, 1) and the row row of shape (3,):

\[ \cmodel{c_{ij}} = \cdata{\mathrm{col}_{i}} + \csecond{\mathrm{row}_{j}} . \]

The column contributes the same value all along row \(i\), and the row the same value all down column \(j\). That is the picture of the ghost copies, written as an index rule.

P02.2.3-F6

The rule applied to four pairs of shapes. Grey 1s are missing axes; the bottom-left pair clashes on the rightmost axis.

Go deeper: why nothing is copied

np.broadcast_to(row, (4, 3)) shows the stretched array without computing anything. Its documentation says it returns a read-only view on the original array, in which more than one element may refer to a single memory location. All four rows of the result are the same three numbers in memory, read four times. That is also why it is read-only: writing into one “copy” would change them all. Lesson P02.3.4 explains views in general.

Worked example

A column plus a row. Take col = [[0], [10], [20], [30]], shape (4, 1), and row = [1, 2, 3], shape (3,). What is col + row?

Line up from the right. (4, 1) over (3,). The missing axis of row counts as 1, so row is treated as (1, 3).

Axis 1 (the right). Sizes 1 and 3. The column has the 1, so it is stretched across to 3 columns.

Axis 0 (the left). Sizes 4 and 1. The row has the 1, so it is stretched down to 4 rows.

Result. Shape (4, 3), with entry \({c_{ij} = \mathrm{col}_i + \mathrm{row}_j}\): row 0 is \({0 + 1,\ 0 + 2,\ 0 + 3}\), which is \(1, 2, 3\), and row 3 is \({30 + 1,\ 30 + 2,\ 30 + 3}\), which is \(31, 32, 33\).

P02.2.3-F3

The worked example. Each input is stretched along its axis of size 1, and the sum is a full (4, 3) table: an addition table of tens and units.

Twelve numbers came from seven stored ones. This “outer” pattern, a column against a row, builds every table of pairs: lesson P02.2.4 uses it for the distances between days.

Code

The rule in plain Python. Padding both shapes with 1s on the left makes them the same length, so that pairing them up from the left is the same as lining them up from the right.

def broadcast_shape(a, b):
    """The shape of a combined with b under NumPy's rule, or a ValueError."""
    n = max(len(a), len(b))
    a = (1,) * (n - len(a)) + tuple(a)      # a missing axis counts as 1
    b = (1,) * (n - len(b)) + tuple(b)
    out = []
    for x, y in zip(a, b):
        if x == y or y == 1:
            out.append(x)
        elif x == 1:
            out.append(y)
        else:
            raise ValueError(f"sizes {x} and {y} clash")
    return tuple(out)

print(broadcast_shape((4, 3), (3,)), broadcast_shape((4, 3), (4, 1)), broadcast_shape((4, 1), (3,)))
print(broadcast_shape((), (4, 3)), broadcast_shape((2, 1, 3), (4, 1)))

(4, 3) (4, 3) (4, 3)
(4, 3) (2, 4, 3)

Now check it against NumPy on every pair of the six shapes of the interactive, errors included.

import numpy as np

shapes = [(), (3,), (4,), (4, 1), (1, 3), (4, 3)]
agree, failures = 0, []
for a in shapes:
    for b in shapes:
        try:
            ours = broadcast_shape(a, b)
        except ValueError:
            ours = "error"
        try:
            theirs = np.broadcast_shapes(a, b)
        except ValueError:
            theirs = "error"
        agree += ours == theirs
        if ours == "error":
            failures.append((a, b))
print(agree, "of", len(shapes) ** 2, "pairs agree")
print(failures)

36 of 36 pairs agree
[((3,), (4,)), ((4,), (3,)), ((4,), (1, 3)), ((4,), (4, 3)), ((1, 3), (4,)), ((4, 3), (4,))]

Six of the 36 pairs fail, and all of them pair a (4,) with something that ends in 3. Now the stretch itself, and the worked example:

row = np.array([1, 2, 3])
col = np.array([[0], [10], [20], [30]])
stretched = np.broadcast_to(row, (4, 3))
print(stretched)
print(np.shares_memory(stretched, row), stretched.flags.writeable)
print(col + row)

[[1 2 3]
 [1 2 3]
 [1 2 3]
 [1 2 3]]
True False
[[ 1  2  3]
 [11 12 13]
 [21 22 23]
 [31 32 33]]

The stretched array shares its memory with row and cannot be written to: there is still only one row of three numbers. Finally the hook, with the error caught so that the lesson keeps running.

temps = np.array([[21, 19, 26], [27, 23, 28], [27, 18, 30], [29, 20, 32]])
print((temps - temps.mean(axis=0)).shape)
try:
    temps - temps.mean(axis=1)
except ValueError:
    print("ValueError: shapes", temps.shape, "and", temps.mean(axis=1).shape, "do not broadcast")

(4, 3)
ValueError: shapes (4, 3) and (4,) do not broadcast

Explore

P02.2-X1

Broadcast stretcher

Pick a shape for A and one for B, then press Step to walk through the rule one axis at a time, or Play to watch it. Try (4,) with (4, 3), then (4, 1) with (4,): which one fails, and what shape does the other give?

Two panels. On the left the shapes (4, 3) and (3,) are lined up from the right, the missing axis counts as 1, and the result is (4, 3). On the right a row of three station means is stretched down into four rows, the three added rows drawn as dashed ghost copies.

Pitfalls

P02.2.3-F4

Wrong beside right. A column of predictions minus a row of targets does not fail: it broadcasts into a (4, 4) table of every prediction minus every target, and the “loss” becomes 4.95 instead of 0.05.

“If the shapes were wrong, NumPy would have told me.”

Only when they clash. A column (4, 1) minus a row (4,) is perfectly legal and gives a (4, 4) table, as on the left of the figure. This is one of the most common silent bugs in machine learning code: a model returns predictions of shape (n, 1), the targets have shape (n,), and the loss is averaged over \(n^2\) wrong numbers. Check shapes with assert pred.shape == y.shape.

“Shapes are lined up from the left.”

From the right. (4, 3) and (4,) share a 4, but the 4 of (4,) is lined up with the 3, so they clash. To line the 4 days up with axis 0, make the day means a column (4, 1).

“Broadcasting copies the smaller array.”

The copies are only a picture. NumPy reads the same stored numbers again for every position along a stretched axis, so broadcasting a row onto a million rows costs no extra memory for the row.

Check yourself

Recall

1. State the broadcasting rule in one sentence.

Show answer

Line the shapes up from the right; on each axis the sizes must be equal or one of them must be 1 (a missing axis counts as 1), and each axis of size 1 is stretched to the other size.

2. What is the shape of temps - temps.mean(axis=0), and why does it work?

Show answer

(4, 3). The means have shape (3,): the 3 matches the stations and the missing axis counts as 1, so the row is stretched down the four days.

3. Why does temps - temps.mean(axis=1) fail?

Show answer

The day means have shape (4,). Lined up from the right, 4 meets the 3 of the stations; they differ and neither is 1.

4. What shape does (4, 1) combined with (3,) give?

Show answer

(4, 3): the column stretches across and the row stretches down.

5. Does np.broadcast_to(row, (4, 3)) store twelve numbers?

Show answer

No. It is a read-only view of the original three numbers, read again for every row.

Exercises

1. ★ Give the result shape, or say which axis fails: (i) (5, 4) with (4,); (ii) (5, 4) with (5,); (iii) (5, 1) with (1, 4); (iv) (3, 1, 4) with (2, 1).

Show solution

(i) (5, 4): 4 matches 4, the missing axis stretches to 5. (ii) Fails on the right-hand axis: 4 against 5. (iii) (5, 4): both arrays stretch. (iv) Lined up from the right, (3, 1, 4) over (1, 2, 1): sizes 4 and 1, then 1 and 2, then 3 and 1, so the result is (3, 2, 4).

2. ★★ Compute np.array([[1], [2], [3]]) * np.array([10, 100]) by hand, giving its shape and every entry.

Show solution

The shapes are (3, 1) and (2,). Lined up from the right: 1 and 2, so the column is stretched to 2 columns; 3 and the missing axis, so the row is stretched to 3 rows. The result has shape (3, 2), with entry \({c_{ij} = \mathrm{col}_i \cdot \mathrm{row}_j}\): [[10, 100], [20, 200], [30, 300]].

3. ★★★ broadcast_shape handles two shapes. Show that applying it twice, broadcast_shape(broadcast_shape(a, b), c), gives np.broadcast_shapes(a, b, c) for a = (6, 7), b = (5, 6, 1) and c = (7,), and explain why the order of the three shapes cannot change the answer when no pair clashes.

Show solution

broadcast_shape((6, 7), (5, 6, 1)): padded, (1, 6, 7) over (5, 6, 1), giving (5, 6, 7). Then with (7,), padded to (1, 1, 7): (5, 6, 7). NumPy’s documentation gives the same example with a fourth shape (5, 1, 7) and also returns (5, 6, 7). On each axis the result is the one size other than 1 that appears there (or 1 if all are 1); the rule succeeds only if all sizes other than 1 on that axis are equal. “The common size other than 1” does not depend on the order in which the sizes are met, so neither does the result.

Summary card

Line the shapes up from the right; stretch every 1.

The anchor figure again: the shapes (4, 3) and (3,) lined up from the right with the result (4, 3), and a row of three station means stretched down into four rows of dashed ghost copies. — P02.2.3-F1
Equal or 1 on every axis.

A missing axis counts as 1; on each axis the sizes must be equal or one must be 1.
The result takes the size that is not 1. (4, 3) with (3,) or (4, 1) gives (4, 3).
(4, 3) with (4,) fails: 3 against 4.
A column (4, 1) and a row (3,) make a (4, 3) table, with nothing copied.

Sources and next steps

Sources. Each was opened and checked against this lesson on 15 November 2026.

NumPy developers, Broadcasting, NumPy user guide, version 2.5. Section “General broadcasting rules”: shapes compared from the trailing dimension, compatible when equal or one of them is 1, missing dimensions assumed to have size one, and the error for shapes (4,3) and (4,). The stretching analogy is described as only conceptual.
NumPy developers, numpy.broadcast_shapes and numpy.broadcast_to, NumPy reference, version 2.5. The first raises a ValueError for shapes that cannot be broadcast and gives the example (5, 6, 7); the second returns a read-only view in which several elements may refer to one memory location.
Jake VanderPlas, Computation on Arrays: Broadcasting, Python Data Science Handbook. Section “Rules of Broadcasting”: the same rule as three steps, padding the shorter shape with ones on its left side.

Next. P02.2.4 Broadcasting at work: centring and scaling uses the rule to centre and standardise the stations and to build a table of distances.

This lesson unlocks

P02.2.4 Broadcasting at work: centring and scaling

Your progress is saved in this browser only. No account, no tracking.