Lecture 20: Central Limit Theorem

What this lecture covers

This lecture completes the course's unit on limit theorems by developing the central limit theorem (CLT) in depth: what it says, what it does and doesn't guarantee, and how to use it in practice. It follows on from the weak law of large numbers and the introduction of the standardized sum Z_n.

The lecture works through the CLT's statement about cumulative distribution functions, its universality across distributions, and its accuracy even for moderate sample sizes, then applies it to polling and binomial calculations, including a refinement using continuity correction. It closes with a puzzle showing where the CLT does not apply. After watching, you should be able to state the CLT precisely, use it to approximate probabilities for sums and binomial random variables, apply the continuity correction, and recognize the assumption (a fixed underlying distribution) that the theorem requires.

Key ideas

• Central limit theorem (CLT): for i.i.d. random variables with finite mean and variance, the standardized sum Z_n = (S_n - n*mu) / (sigma*sqrt(n)) has a cumulative distribution function that converges to that of a standard normal random variable, regardless of the original distribution's shape.
• Statement about CDFs, not PMFs/PDFs: the CLT guarantees convergence of cumulative distributions, not of probability mass or density functions, though for many practical cases (like the binomial) the PMF approximation is also good.
• Universality: only the mean and variance of the underlying distribution are needed to apply the CLT; the shape of that distribution doesn't matter.
• Practical accuracy at small n: the normal approximation is often good even for n around 15-30, though how fast it kicks in depends on how close the original distribution already is to symmetric/normal-shaped.
• Continuity correction: when approximating a discrete sum (like a binomial) with a normal distribution, using the midpoint between integers (e.g., 21.5 instead of 21 or 22) gives a substantially more accurate approximation.
• Two different limits for the binomial: fixing p and letting n -> infinity gives the normal approximation (CLT); letting n -> infinity while p -> 0 with np fixed gives the Poisson approximation instead. The two are valid in different regimes, roughly split at np around 10-20.
• The CLT requires a fixed distribution: it applies to sums of variables drawn repeatedly from one fixed distribution; the theorem does not apply if the distribution of each term changes as more terms are added.

Walkthrough

Restating the central limit theorem (0:00)

The lecture recaps that summing i.i.d. random variables and standardizing the result (subtracting the mean, dividing by the standard deviation) produces Z_n, whose CDF converges to the standard normal CDF as n -> infinity, regardless of the underlying distribution.

What the theorem does and doesn't promise (3:04)

The lecture stresses the CLT's universality (only mean and variance matter) and its role as a computational shortcut, contrasts it with the intuition behind Brownian motion and noisy phenomena built from many small independent effects, and demonstrates with plotted examples that two very different discrete distributions can have visibly different summed PMFs but nearly identical CDFs, since the theorem is fundamentally a statement about CDFs.

How large n needs to be in practice (11:18)

Using plots of sums of discrete uniform and truncated geometric distributions, the lecture shows the normal approximation becoming visually accurate by n around 20-32, faster for symmetric starting distributions than for skewed ones.

Revisiting the polling problem (14:23)

The lecture reapplies the CLT to the earlier polling example, standardizing the accuracy/confidence event into a statement about Z_n, using the conservative bound sigma <= 1/2 for a Bernoulli proportion, and computing that a sample of about 9,600 achieves 95% confidence within 1 percentage point, versus the looser 50,000 bound Chebyshev's inequality gave earlier.

Applying the CLT to a binomial calculation (27:49)

Working a concrete binomial example (n=36, p=1/2), the lecture computes the mean and standard deviation, standardizes the event S_n < 21, and reads off a normal-table approximation, then compares it to the exact binomial answer to show the approximation is decent but not excellent.

The continuity correction (32:02)

The lecture shows that because S_n is discrete, the events S_n <= 21, S_n < 22, and S_n < 21.5 are mathematically identical but yield different normal approximations; integrating up to the midpoint 21.5 gives a much more accurate result, and the same half-unit trick lets you approximate an individual probability like P(S_n = 19) by integrating the normal curve between 18.5 and 19.5.

A puzzle: when the CLT doesn't apply (42:28)

The lecture presents a Poisson process split into n small sub-intervals, each approximately Bernoulli, and asks why naively applying the CLT would (wrongly) suggest the Poisson-distributed total is normal. The resolution is that the CLT assumes a fixed underlying distribution for each term; here, the per-interval distribution changes as n grows, so the theorem's hypotheses aren't met. The lecture closes by contrasting this Poisson-limit regime with the fixed-p regime where the normal approximation is valid, giving np > 10-20 as a rough rule of thumb for preferring the normal approximation.

Before you watch

• Watch the previous lecture on the weak law of large numbers, which introduces the Markov and Chebyshev inequalities and the standardized sum Z_n.
• Review the binomial and Poisson distributions and normal distribution tables.

Check your understanding

1. Why is the central limit theorem formally a statement about cumulative distribution functions rather than probability mass or density functions?
2. How does the continuity correction improve the normal approximation to a binomial probability, and why does using the midpoint work better than using either integer endpoint?
3. Why does naively applying the CLT to the sub-intervals of a Poisson process lead to a contradiction, and what assumption does this violate?
4. Under what conditions would you prefer the Poisson approximation to the binomial over the normal approximation?