Lecture 3: Independence

What this lecture covers

After two lectures on axioms and conditional probability, this lecture answers a natural question: when does knowing that one event occurred tell you nothing about another? The answer is the concept of independence, one of the most important — and most frequently misunderstood — ideas in probability.

The lecture starts with a quick review of the multiplication rule, total probability theorem, and Bayes' rule from the previous lecture using a three-coin-toss model, then builds up the formal definition of independence, carefully separates it from the unrelated idea of disjointness, extends it to conditional independence and to collections of more than two events, and closes with the well-known "king and his sibling" puzzle to show how sensitive probability answers are to hidden assumptions.

Key ideas

• Independence, intuitively: two events are independent if learning that one occurred doesn't change your beliefs about the other.
• Formal definition: A and B are independent if P(A ∩ B) = P(A) · P(B), which is preferred over the conditional-probability version because it also applies when an event has probability zero.
• Disjoint is not the same as independent: disjoint events are actually strongly dependent, since learning one occurred tells you the other definitely did not.
• Conditional independence: independence can hold or fail to hold once you condition on some other event C, even if it held (or failed) unconditionally.
• Pairwise vs. mutual independence: events can be independent two at a time without the whole collection being independent together — an extra condition involving all events jointly is required.
• Independence of multiple events: a full set of events is independent only if every sub-collection's joint probability factors as the product of individual probabilities.

Walkthrough

Independence as "no new information" (11:14)

After reviewing the multiplication, total probability, and Bayes' rules using a three-coin-toss model, the lecture points out a special feature of that model: the conditional probability of heads on a later toss is the same, p, regardless of what happened on earlier tosses. This observation is used to motivate the intuitive meaning of independence — an event conveys no information about another if knowing it occurred leaves your beliefs about the other unchanged.

Formal definition and zero-probability events (14:23)

The intuitive "conditional probability doesn't change" idea is translated into the definition P(A ∩ B) = P(A) · P(B). This version is preferred over one based on conditional probability because conditional probability requires positive probability in the conditioning event, while the product definition works even when one event has probability zero — in which case it is automatically independent of every other event, a fact that can feel odd but follows directly from the definition.

Independence versus disjointness (18:34)

A common confusion is addressed directly: two disjoint events are not independent — they are "as dependent as Siamese twins," because if one occurs, the other is certain not to. The lecture stresses that a Venn diagram alone can never establish independence; you must check the numbers, since independence depends on the actual probabilities assigned, not just the geometric layout of events.

Conditional independence and the two-coin example (24:46)

Independence can be defined within a conditional universe too: A and B are conditionally independent given C if P(A ∩ B | C) = P(A|C) · P(B|C). The lecture shows with a picture that unconditional independence does not survive conditioning in general. It then works a richer example: two biased coins (one favoring heads, one favoring tails) where you first pick a coin at random and then flip it repeatedly. Within each conditional sub-model (given which coin was picked), flips are independent, but across the whole model they are not — observing many heads in a row makes you revise your belief about which coin was chosen, which then changes your prediction for future flips.

Independence of multiple events and pairwise independence (30:55)

Extending the definition to more than two events requires more than pairwise checks. The lecture gives an example with two independent fair coin tosses and a third event ("both tosses match"), showing that all three pairs of events are independent on their own, yet the three events together are not, because knowing both of the first two events determines the third with certainty. Full independence of a collection requires that the probability of every sub-collection's intersection equal the product of the individual probabilities.

The king and his sibling (41:07)

The lecture closes with a puzzle: a royal family has two children and we're told at least one is a boy (the king); what is the probability the other child is a girl? Modeling the four equally likely outcomes for two children and removing the "girl, girl" possibility gives an answer of 2/3, not the naive 1/2. But the lecture cautions that this answer depends on unstated assumptions about how the family came to have two children — different scenarios (having children until a boy is born, or always favoring one gender) can lead to different, equally valid answers, illustrating how loosely worded probability problems hide critical assumptions.

Before you watch

• Watch "Lecture 2: Conditioning and Bayes' Rule" first — this lecture relies on the multiplication rule, total probability theorem, and Bayes' rule.
• Be comfortable with the definition of conditional probability and basic probability-tree diagrams.

Check your understanding

1. Why is the product-based definition of independence, P(A ∩ B) = P(A) · P(B), preferred over a conditional-probability definition?
2. Why are two disjoint events with positive probability never independent?
3. In the two-biased-coins example, why does observing a long run of heads change the probability of the next toss being heads, even though flips within one coin are independent?
4. What extra condition is needed for three events to be mutually independent, beyond each pair being independent?
5. Why does the answer to the king-and-sibling puzzle depend on the family's (unstated) strategy for having children?