value = 1.0 # the starting value a
for step in range(10): # ten steps ...
value = value * 2 # ... each one multiplies by the factor b = 2
print(value) # 1024.0, the end of the doubling sequence1024.0Exponential growth and decay
A line adds the same amount at every step; an exponential multiplies by the same factor. Meet the exponential function, its growth and decay, doubling times and half-lives read from a table, and the number e as the value that compounding in ever more steps settles towards.
M01.4.2L0 Foundation25 minIn review
Before this lesson
M01.4.1 Powers and their rules M01.3.2 Straight lines: slope and intercept
After this lesson you can
- Tell linear from exponential change in a table: equal differences against equal ratios.
- Write the rule of exponential growth or decay from a starting value and a factor per step, and evaluate it.
- Read a doubling time or a half-life from a table of values, and explain why it is the same from every starting point.
- Describe the number e as the value that compounding in more and more steps settles towards, and compute powers of e in Python.
In review. The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.
The question
A training run starts with a learning rate of 0.1 and multiplies it by 0.9 after every pass through the data, which is called an epoch. After how many epochs is the rate below half of where it started? When is it below 0.01? And does it ever reach zero?
From the last lesson
1. What is \({10^{-3}}\) as a fraction and as a decimal?
Show answer
\({\frac{1}{10^3} = \frac{1}{1000} = 0.001}\).
2. State the product rule of powers and say why it is true.
Show answer
\({b^m\, b^n = b^{m+n}}\). The product is a row of \(m\) factors of \(b\) followed by \(n\) more, so it has \({m + n}\) factors.
Intuition
A quantity can change step by step in two simple ways. It can add the same amount at every step, as 1, 6, 11, 16 adds 5. Or it can multiply by the same factor at every step, as 1, 2, 4, 8 multiplies by 2. The first is a line (M01.3.2). The second is an exponential.
A line adds the same amount at each step; an exponential multiplies by the same factor. The line starts ahead, but from step 5 on the exponential is ahead, and the gap keeps growing.
The line starts ahead: after four steps it is at 21 and the exponential only at 16. Then the exponential overtakes, at step 5, and pulls away. The reason is in the steps themselves. The line’s step is always +5. The exponential’s step is +1, +2, +4, +8, +16, +32: each step is double the one before, because each step adds a copy of everything there already is.
That gives a quick test for a table of numbers. Equal differences between neighbours mean a line. Equal ratios between neighbours mean an exponential.
Build it up
Step 1. The rule as a formula. Start at \(a\) and multiply by \(b\) at every step. After \(x\) steps there are \(x\) factors of \(b\):
\[ f(x) = a\, b^x . \]
At the start, \({f(0) = a\, b^0 = a}\), because \({b^0 = 1}\) (M01.4.1). So \(a\) is the starting value and \(b\) is the growth factor. The doubling sequence of the course is \({a = 1}\), \({b = 2}\): \(2^0, 2^1, \dots, 2^{10}\) is 1, 2, 4, …, 1024.
Step 2. Growth and decay. Take the starting value \(a\) positive, as it is for an amount of money or a learning rate. When \(b\) is bigger than 1, each step makes the value bigger: exponential growth. When \(b\) is between 0 and 1, each step makes it smaller: exponential decay. When \({b = 1}\) nothing changes, so that case is left out, and \(b\) is kept positive so that the value does not flip sign at every step.
Step 3. Between the steps. Lesson M01.4.1 gave meaning to fractional exponents, so \(b^x\) makes sense for every \(x\): \({2^{0.5} = \sqrt2 \approx 1.4142}\) is the value at \({x = 0.5}\), halfway between the steps \({x = 0}\) and \({x = 1}\). The dots of a table join into a smooth curve.
Growth when \({b > 1}\), decay when \({0 < b < 1}\). Every curve \(b^x\) passes through \((0, 1)\), and each one comes ever closer to the horizontal axis without touching it.
Step 4. Percentages. Growth by 5% a step keeps 100% and adds 5%, so it multiplies by \({b = 1.05}\). Decay by 10% a step keeps 90%, so it multiplies by \({b = 0.9}\). In general a rate \(r\) gives \({b = 1 + r}\) for growth and \({b = 1 - r}\) for decay.
Step 5. Doubling time and half-life. The doubling time is the number of steps the quantity takes to double; the half-life is the number it takes to halve. Both can be read from a table. For 5% growth:
| steps \(t\) | \(1.05^t\) |
|---|---|
| 13 | 1.8856 |
| 14 | 1.9799 |
| 15 | 2.0789 |
| 16 | 2.1829 |
The value passes 2 between 14 and 15 steps, so the doubling time is a little over 14 steps. The next lesson finds it exactly.
Step 6. The number e. Growth of 100% over one period doubles the amount: 1 becomes 2. Split the period into two halves with 50% growth each, and \({1.5^2 = 2.25}\). Four quarters of 25% each give \({1.25^4 \approx 2.4414}\). In general, \(n\) steps of growth \(\frac1n\) each give \({(1 + \frac1n)^n}\).
More steps give more growth, but not without limit: the staircases climb towards a smooth curve whose end is \({e \approx 2.7183}\).
More steps give a bigger result, but the results level off:
| steps \(n\) | \({(1 + \frac1n)^n}\) |
|---|---|
| 1 | 2 |
| 2 | 2.25 |
| 4 | 2.4414 |
| 12 | 2.6130 |
| 365 | 2.7146 |
| 1,000 | 2.7169 |
| 100,000 | 2.7183 |
The number they settle towards is called \(e\). It is about 2.718281828, and like \(\sqrt2\) it is not a fraction. What “settle towards” means exactly is a limit, which is the subject of M04.1; here the table is our evidence.
The formal statement
Three results carry the rest of the module. Each is derived from the rules of M01.4.1. Take \({a > 0}\) throughout.
Result 1: equal steps multiply by equal factors. Move the input on by any amount \(h\). The product rule splits the power in two:
\[ \begin{aligned} f(x + h) &= a\, b^{x+h} \\ &= a\, b^x\, b^h \\ &= b^h\, f(x) . \end{aligned} \]
With \({h = 1}\), each whole step multiplies by \(b\), so the ratio of neighbours \({f(x + 1) / f(x)}\) is always \(b\). Compare a line \({g(x) = mx + c}\) (the intercept is called \(c\) here, because \(b\) is in use): there the difference of neighbours \({g(x + 1) - g(x)}\) is always \(m\).
Result 2: the doubling time is the same from every starting point. Suppose \({b^d = 2}\). Result 1 with \({h = d}\) gives \({f(x + d) = 2 f(x)}\) for every \(x\). So the quantity doubles every \(d\) steps, whether it is small or large. In the same way, if \({b^h = \tfrac12}\) it halves every \(h\) steps, and \(h\) is the half-life.
Result 3: always positive, never zero. A positive base raised to any power is positive (M01.4.1), so \({f(x) = a\, b^x > 0}\) for every \(x\). The graph passes through \((0, a)\), and on the side where it shrinks it gets as close to the horizontal axis as you like, but every step still multiplies by a positive factor, so it never arrives.
The number e. The number \({e \approx 2.7183}\) is the value that \({(1 + \frac1n)^n}\) settles towards as \(n\) grows. The exponential with base \(e\) is written \(e^x\) or \(\exp(x)\); the second form is easier to read when the exponent is long.
Machine learning uses both kinds of exponential every day. Training libraries offer a learning rate that decays by a constant factor per epoch, as in the question above. And the sigmoid of ML01.6, which turns any score \(z\) into a probability, is built from \({e^{-z}}\):
\[ \sigma(z) = \frac{1}{1 + e^{-z}} . \]
Go deeper: e to any power, by the same table
Compounding growth \(x\) in \(n\) steps gives \({(1 + \frac{x}{n})^n}\), and this settles towards \(e^x\). For \({x = 2}\): with \({n = 10}\) steps it is about 6.1917, with \({n = 1000}\) about 7.3743, and with \({n = 100{,}000}\) about 7.3889, while \({e^2 \approx 7.3891}\). The power-of-a-power rule shows why. For a positive \(x\), write \({m = \frac{n}{x}}\), so that \({\frac{x}{n} = \frac{1}{m}}\) and \({n = mx}\):
\[ \left(1 + \tfrac{x}{n}\right)^n = \left[\left(1 + \tfrac{1}{m}\right)^m\right]^x . \]
As \(n\) grows so does \(m\), and the inside settles towards \(e\), so the whole settles towards \(e^x\). M04.1 makes this exact, and M01.9.3 runs the same table with an imaginary rate.
Worked example
A learning rate starts at \({\eta_0 = 0.1}\) and is multiplied by 0.9 after every epoch. When is it first below half its start, and when below 0.01? The counter \(t\) starts at 0, so \(\eta_0\) is the rate before any epoch and \(\eta_t\) is the rate after \(t\) epochs. The rule is an exponential with starting value 0.1 and factor 0.9, which is decay by 10% per epoch:
\[ \eta_t = 0.1 \times 0.9^t . \]
The worked example as a trace. Top: the first three epochs by hand. Bottom: the whole schedule; the rate first drops below 0.05 at \({t = 7}\) and below 0.01 at \({t = 22}\).
By hand. Each epoch multiplies the previous rate by 0.9, so the rate goes 0.1, then 0.09, then 0.081, then 0.0729. Further on, the formula is quicker than the chain (values after \({t = 3}\) are rounded to four decimal places):
| epoch \(t\) | rate \(\eta_t\) |
|---|---|
| 0, 1, 2, 3 | 0.1, 0.09, 0.081, 0.0729 |
| 6 | 0.0531 |
| 7 | 0.0478 |
| 21 | 0.0109 |
| 22 | 0.0098 |
Half the start. Half of 0.1 is 0.05. After 6 epochs the rate is still about 0.0531; after 7 it is about 0.0478. So it first drops below half after 7 epochs, and the half-life lies between 6 and 7 epochs.
Below 0.01. After 21 epochs the rate is about 0.0109; after 22 it is about 0.0098. So it first drops below 0.01 after 22 epochs.
Ever zero? No. By Result 3, \({0.1 \times 0.9^t}\) is positive for every \(t\): each epoch takes away only a tenth of what is left.
Code
Growth by a loop: start at \(a\) and multiply by \(b\) once per step.
The learning-rate schedule of the worked example, run until the rate falls below 0.01. A while loop repeats for as long as its condition holds (P01.3.2):
lr = 0.1 # the rate before any epoch (t = 0)
epoch = 0
while lr >= 0.01: # repeat while the rate is still 0.01 or more
lr = lr * 0.9 # one epoch: multiply by 0.9
epoch = epoch + 1 # and count it
print(epoch, round(lr, 4)) # 22 0.0098
print(round(0.1 * 0.9 ** 7, 4)) # the formula at t = 7: 0.047822 0.0098
0.0478The table that leads to \(e\):
for n in [1, 2, 4, 12, 365, 1000, 100000]:
print(n, round((1 + 1 / n) ** n, 4)) # n steps, each one growth 1/n1 2.0
2 2.25
4 2.4414
12 2.613
365 2.7146
1000 2.7169
100000 2.7183The library versions. The math module holds \(e\) and the function exp, and NumPy’s exp works on a whole array at once (a first look at P02):
import math
import numpy as np
print(math.e) # 2.718281828459045
print(math.exp(1), math.exp(2)) # e to the 1 and e to the 2
print(np.exp(np.array([0.0, 1.0, 2.0]))) # all three powers at once
z = 2.0
print(round(1 / (1 + math.exp(-z)), 4)) # the sigmoid of ML01.6 at z = 2: 0.88082.718281828459045
2.718281828459045 7.38905609893065
[1. 2.71828183 7.3890561 ]
0.8808Explore
This module’s interactive, the log ruler, lives in lesson M01.4.4. On its log ruler each equal step multiplies by the same factor, which is exactly what an exponential does at every step.
Pitfalls
Wrong beside right. Adding 5 each step reaches 150. Multiplying by 1.05 each step reaches 162.89, because each new 5% is taken of a bigger amount.
“Growth of 5% a step for 10 steps is 50% growth.”
That would be true if each step added 5% of the starting amount. Exponential growth adds 5% of the current amount, so the steps grow: \({1.05^{10} \approx 1.6289}\), which is 62.89% growth.
“\(x^2\) and \(2^x\) are the same kind of function.”
In \(x^2\) the input is the base and the exponent is fixed: that is the parabola of M01.3.3. In \(2^x\) the base is fixed and the input is the exponent. For positive \(x\) they meet at \({x = 2}\) and \({x = 4}\), but from \({x = 5}\) on, \(2^x\) is ahead at every whole number: one more step doubles \(2^x\), while it multiplies \(x^2\) by \({(\frac{x + 1}{x})^2}\), which is at most \(1.44\) once \({x \ge 5}\).
“Down 10% and then up 10% is back where it started.”
The factors multiply: \({0.9 \times 1.1 = 0.99}\), so the result is 1% below the start. Percentage changes compound; they do not cancel.
“A learning rate that decays for long enough reaches zero.”
Not in mathematics: every value \({0.1 \times 0.9^t}\) is positive. A computer is another matter, since it can store only so many digits; lesson M01.4.4 shows a product that does reach 0.0, and the cure.
Check yourself
Recall
1. A table reads 3, 6, 12, 24. Is the change linear or exponential, and what is the rule?
Show answer
Exponential: each ratio of neighbours is 2, so \({f(x) = 3 \times 2^x}\), counting \(x\) from 0.
2. For \({f(x) = 5 \times 0.8^x}\), what are the starting value and the growth factor, and is this growth or decay?
Show answer
The starting value is 5 and the factor is 0.8. The factor lies between 0 and 1, so it is decay.
3. Why does the graph of \(2^x\) never touch the horizontal axis?
Show answer
Because \(2^x\) is positive for every \(x\): a positive base raised to any power is positive. It comes as close to 0 as you like on the left, but never reaches it.
4. What factor per step is growth of 7%? Decay of 7%?
Show answer
1.07 for growth and 0.93 for decay.
5. What number does \({(1 + \frac1n)^n}\) settle towards as \(n\) grows, and what is it called?
Show answer
About 2.7183. It is called \(e\).
Exercises
1. ★ A colony of bacteria starts at 500 and triples every hour. Write the rule and find the count after 4 hours.
Show solution
The starting value is 500 and the factor is 3, so \({f(t) = 500 \times 3^t}\) after \(t\) hours. After 4 hours:
\[ 500 \times 3^4 = 500 \times 81 = 40{,}500 . \]
2. ★★ A medicine in the blood falls by 20% per hour from 200 mg. Write the rule, and use a table to find after how many whole hours it first drops below 100 mg.
Show solution
A fall of 20% keeps 80%, so the factor is 0.8 and \({f(t) = 200 \times 0.8^t}\). Multiplying by 0.8 each hour gives 200, 160, 128, 102.4, 81.92. After 3 hours there are still 102.4 mg; after 4 hours 81.92 mg. So it first drops below 100 mg after 4 hours, and the half-life lies between 3 and 4 hours.
3. ★★★ You want a learning rate to fall from 0.1 to 0.001 in exactly 100 epochs, multiplying by the same factor \(b\) every epoch. Find \(b\) to four decimal places.
Show solution
After 100 epochs the rate is \({0.1 \times b^{100}}\), and this must be 0.001. Dividing by 0.1 gives \({b^{100} = 0.01 = 10^{-2}}\). Take the \(\frac{1}{100}\) power of both sides and use the power-of-a-power rule:
\[ \begin{aligned} b &= \left(10^{-2}\right)^{1/100} \\ &= 10^{-2/100} \\ &= 10^{-0.02} \approx 0.9550 . \end{aligned} \]
The rate loses about 4.5% per epoch. Check: \({0.955^{100} \approx 0.0100}\).
Summary card
An exponential multiplies by the same factor at every step.
A line adds; an exponential multiplies.
\[ \begin{aligned} f(x) &= a\, b^x \\ f(x + h) &= b^h\, f(x) \\ e &\approx 2.7183 \end{aligned} \]
- Growth when \({b > 1}\), decay when \({0 < b < 1}\); always positive, never 0.
- Doubling time \(d\): \({b^d = 2}\). Half-life \(h\): \({b^h = \frac12}\). The same from any start.
- \(e^x\) is also written \(\exp(x)\):
math.expandnumpy.exp.
Sources and next steps
Sources. Each was opened and checked against this lesson on 12 November 2026.
- Jay Abramson and others, Algebra and Trigonometry 2e, OpenStax. Section 6.1, Exponential Functions: \({f(x) = ab^x}\) with initial value \({a \neq 0}\) and \({b > 0}\), \({b \neq 1}\); growth for \({b > 1}\) and decay for \({0 < b < 1}\); linear growth by the same amount against exponential growth by the same percentage; \(e\), which it calls an irrational number, as the value of \({(1 + 1/n)^n}\) as \(n\) increases without bound, with a table that starts 2, 2.25, 2.441406, 2.613035. Section 6.2: the graph of \(b^x\) passes through \((0, 1)\) with the horizontal asymptote \({y = 0}\). Section 6.7: half-life and doubling time.
- Python Software Foundation,
math.expandmath.e, Python 3.12 documentation; NumPy,numpy.exp, NumPy 2.5 manual, which computes the exponential element by element and notes that \(e^x\) is positive for real input. - PyTorch,
ExponentialLR, PyTorch 2.15 documentation: it “decays the learning rate of each parameter group by gamma every epoch”, gamma being the multiplicative factor.
Next. M01.4.3 Logarithms: the exponent as the answer runs the exponential backwards and finds the doubling time exactly.
This lesson unlocks
M01.4.3 Logarithms: the exponent as the answer M01.7.1 Sequences: lists with a rule
Your progress is saved in this browser only. No account, no tracking.