Lecture 2: Conditioning and Bayes' Rule

What this lecture covers

This lecture answers a central question: how should probabilities change once you learn partial information about an experiment's outcome? It builds directly on the axioms and sample-space ideas from the first lecture, and introduces conditional probability as the tool for revising beliefs.

After a quick review that also repairs a subtle gap from the previous lecture, the lecture develops conditional probability from a numerical example to a formal definition, then shows three uses of it: computing probabilities of combined events (the multiplication rule), breaking a probability into scenarios (the total probability theorem), and reversing the direction of conditioning to make inferences (Bayes' rule). By the end you can build a probability model out of conditional probabilities and use it to answer both "what's likely to happen" and "given what I observed, what probably caused it" questions.

Key ideas

• Conditional probability: P(A|B) = P(A ∩ B) / P(B), the fraction of B's probability that also belongs to A; undefined when P(B) = 0.
• Conditioning creates a new probability law: once you condition on B, the conditional probabilities on the remaining sample space still satisfy all the usual axioms.
• Countable additivity vs. uncountable unions: the additivity axiom applies to sequences of disjoint events, not to unions over uncountably many points — illustrated by a paradox where summing zero-probability points would wrongly give 1 = 0.
• Multiplication rule: the probability that several events all occur equals the product of a chain of conditional probabilities, one for each step down a probability tree.
• Total probability theorem: if a sample space is split into a partition of scenarios, the probability of any event is a weighted average of its probability under each scenario.
• Bayes' rule: reverses conditioning — given the observed effect, it recovers the probability of each possible cause by combining the total probability theorem with the definition of conditional probability.
• Base-rate effect: even a reliable detector can give a low probability of being correct if the event it is detecting is rare, because false alarms accumulate.

Walkthrough

Review and the countable additivity subtlety (4:07)

The lecture opens with a fast recap of sample spaces, events, and the three axioms, then revisits the unit-square example from the previous lecture. Trying to write the whole square as a union of single points and apply the additivity axiom produces the false conclusion 1 = 0, because that axiom only applies to sequences of disjoint events, and the points of a square cannot be arranged into a sequence — the square is an uncountable set. This motivates a reminder that individual points can have zero probability while "impossible" is a stronger statement than "probability zero"; in continuous models, some zero-probability outcome always occurs.

Conditional probability: definition and revising beliefs (12:23)

Using a sample space split into regions with given probabilities, the lecture asks how those probabilities should change once you're told the outcome lies in some event B. Intuitively, the relative proportions of probability inside B should be preserved, which leads to the formal definition P(A|B) = P(A ∩ B) / P(B). The lecture checks this definition against the intuitive answer and notes the symmetric identity P(A ∩ B) = P(B) · P(A|B) = P(A) · P(B|A), interpreted as "the fraction of experiments where B happens, and among those, the fraction where A also happens."

Applying conditioning to the dice example (20:42)

Returning to the two-dice sample space from the first lecture, the lecture conditions on the event that the minimum of the two rolls is 2, then asks for the probability that the maximum equals 1 or 2 within that reduced sample space. It shows that when the original distribution is uniform, the conditional distribution over the reduced (conditioning) event is still uniform, so conditional probabilities can often be found by simple counting rather than by working through the full ratio formula.

The radar example: building a model from conditionals (24:48)

A model of an airplane detector is built directly out of conditional probabilities: there is a 5% chance a plane is present, and given that, a 99% chance the radar detects it; given no plane, a 10% chance of a false alarm. The lecture shows how to recover an ordinary joint probability (plane present and detected) by multiplying along a probability tree, and then computes the overall probability that the radar registers something by adding the two ways that can happen. Finally it computes, given that the radar registered something, the probability a plane is actually there — about 34%, lower than the radar's stated reliability would suggest, because false alarms are common relative to true detections.

Three tools: multiplication rule, total probability, Bayes' rule (33:01)

The lecture generalizes the radar example into three reusable tools. The multiplication rule finds the probability of a chain of events by multiplying conditional probabilities along a tree, proved by repeated use of the definition of conditional probability. The total probability theorem finds the probability of an event B by summing its probability under each scenario in a partition, weighted by the probability of that scenario. Bayes' rule then combines these two to reverse the order of conditioning: starting from a "cause to effect" model (P(B|A_i)), it computes "effect to cause" probabilities (P(A_i|B)), which is the basis of inference from observed data back to underlying causes.

Before you watch

• Watch "Lecture 1: Probability Models and Axioms" first — this lecture assumes familiarity with sample spaces, events, and the three axioms.
• Be comfortable with basic algebra and simple probability-tree diagrams.

Check your understanding

1. Why does the additivity axiom fail when applied to an uncountable union of single-point events?
2. How is the conditional probability P(A|B) different from the unconditional probability P(A), and when are they equal?
3. In the radar example, why is the probability of a plane being present given a detection so much lower than the radar's stated 99% accuracy?
4. How does the total probability theorem let you compute P(B) without directly examining the whole sample space at once?
5. What is the difference between a "cause to effect" conditional probability and the "effect to cause" probability that Bayes' rule computes?