Seyed Masoud Hosseini · Overview · Study log · Weekly summaries · Ideas · Search · Transcript · RSS feed

Probability · Lecture 25 of 76 · 13:45

Joint Probability Mass Function (PMF) Drill 2

Joint Probability Mass Function (PMF) Drill 2 on YouTube

Study guide

What this lecture covers

This recitation problem builds further fluency with joint PMFs, following on from an earlier drill. Starting from a joint PMF of X and Y plotted on a grid, with some probability masses left as unknowns, it works through marginalization, sketching a conditional PMF, computing a conditional expectation, and reasoning about independence directly from the structure of the table rather than by formula.

By the end, you'll have a fast, intuitive method for computing conditional PMFs, and a way to test or enforce independence (including conditional independence) by comparing relative frequencies across different conditioning values.

Key ideas

  • Marginalization: P(X=x) is found by summing the joint PMF over all values of y, or equivalently by adding the probabilities of the disjoint outcomes where x occurs.
  • Fast conditional PMF trick: to condition on X=x, sum the numerators of the joint probabilities in that slice to get a new denominator, then keep the original numerators over that new denominator.
  • Conditional expectation via symmetry: for a symmetric conditional PMF, the expectation is just the center of mass, no formula needed.
  • Testing independence from a table: if conditioning on different values of x changes the relative frequencies of y, then x tells you something about y, so X and Y cannot be independent, regardless of unknown probability values.
  • Zero-probability entries matter: an outcome with probability zero changes which values of y are even possible under a given x, which alone can rule out independence.
  • Enforcing conditional independence: given that two variables are independent within a restricted event, the relative frequencies of y across different x values (within that event) must match, which pins down otherwise unknown probabilities.

Before you watch

  • Be comfortable with joint PMFs, marginal PMFs and conditional PMFs of discrete random variables.
  • Know the definition of independence for random variables from earlier in the course.
  • This drill assumes you've seen the "Joint Probability Mass Function (PMF) Drill 1" problem or equivalent material on joint PMFs.

Check your understanding

  1. Why does a single zero-probability entry in a joint PMF table make it possible to rule out independence without knowing any other value?
  2. Explain the shortcut method for computing a conditional PMF in your own words.
  3. In part f, why does the answer to an earlier part (the unconditional joint probability) get reused to compute the conditional probability given event B?
  4. How would you check whether two random variables are conditionally independent given some event, versus independent overall?

Vocabulary

drill (noun)
A short practice exercise that trains one specific skill.
This drill focuses on reading joint PMF tables quickly.
fluency (noun)
The ability to do something smoothly and without much effort, from practice.
The exercise builds fluency with joint PMF tables.
grid (noun)
A pattern of rows and columns used to organize numbers or values.
The joint PMF is plotted on a grid of x and y values.
marginalization (noun)
The process of finding one variable's overall probability by adding over all values of another variable.
Marginalization gives P(X=x) by summing the joint PMF over y.
sketch (verb)
To draw or describe something roughly, without full detail.
We sketch the conditional PMF to see its shape.
conditional expectation (phrase)
The average value of a variable, calculated only within a specific known condition.
The conditional expectation is easy to find using symmetry.
symmetric (adjective)
Having the same shape or pattern on both sides of a center point.
A symmetric conditional PMF has its expectation at the center.
center of mass (phrase)
The balancing point of a distribution of weights or probabilities.
For a symmetric PMF, the expectation is just the center of mass.
relative frequency (phrase)
How often something happens compared to the total number of cases.
Comparing relative frequencies across x values can rule out independence.
rule out (phrasal verb)
To show that something is not possible.
A single zero entry can rule out independence.
unknown (adjective)
Not yet known or given a value.
Some probability masses in the table are left as unknowns.
denominator (noun)
The bottom number in a fraction.
Summing the numerators in a slice gives the new denominator.
numerator (noun)
The top number in a fraction.
The original numerators stay the same over the new denominator.
pin down (phrasal verb)
To find or fix an exact value for something uncertain.
The independence condition pins down the missing probabilities.
enforce (verb)
To make sure a rule or condition is true.
We enforce conditional independence to solve for the unknowns.
joint PMF (phrase)
A table or formula giving the probability that two variables each take a specific value at the same time.
The problem starts from a joint PMF plotted on a grid.
incomplete (adjective)
Missing one or more parts, not fully filled in.
The joint PMF table is incomplete, with some values left unknown.
structure (noun)
The way the parts of something are arranged and connected.
Independence is reasoned about from the structure of the table.
disjoint (adjective)
Not overlapping; unable to happen at the same time.
Marginalization adds probabilities across disjoint outcomes.
outcome (noun)
A specific possible result of a random process.
Each cell in the table represents one possible outcome.
reason about (phrase)
To think through and draw conclusions about a situation.
We reason about independence directly from the table.
reuse (verb)
To use something again instead of calculating it from scratch.
An earlier answer is reused to compute the conditional probability.
formula (noun)
A fixed rule expressed with symbols for calculating something.
The fast method finds a conditional PMF without a full formula.
regardless of (phrase)
Without being affected by or depending on something.
Independence fails regardless of the unknown probability values.
assume (verb)
To accept something as true without proof, for the sake of an argument.
We assume conditional independence within the restricted event.
intuitive (adjective)
Easy to understand naturally, without a long formal derivation.
The drill gives a fast, intuitive method for conditional PMFs.

Chapters

From the YouTube description

MIT 6.041SC Probabilistic Systems Analysis and Applied Probability, Fall 2013
View the complete course: http://ocw.mit.edu/6-041SCF13
Instructor: Katie Szeto

License: Creative Commons BY-NC-SA
More information at http://ocw.mit.edu/terms
More courses at http://ocw.mit.edu

← Lecture 7: Discrete Random Variables III · Lecture 8: Continuous Random Variables →