Seyed Masoud Hosseini · Overview · Study log · Weekly summaries · Ideas · Search · Transcript · RSS feed

Probability · Lecture 63 of 76 · 13:36

Convergence in Probability and in the Mean, Part 1

Convergence in Probability and in the Mean Part 1 on YouTube

Study guide

What this lecture covers

This is a worked problem comparing two sequences of random variables, X_n and Y_n, that both place most of their mass near 0 but differ in what happens on rare events. It asks whether each sequence converges in probability, using the Chebyshev inequality where it works and the distribution directly where the inequality is too weak, and closes by testing whether convergence in probability implies convergence of expected values.

It follows directly from the course's treatment of the Chebyshev inequality and convergence in probability. After watching, you should be able to compute expected value and variance for such sequences, apply Chebyshev's inequality to prove convergence in probability, recognize when that inequality is uninformative, and explain why convergence in probability does not guarantee convergence in expectation.

Key ideas

  • Two constructed sequences: X_n takes value 1 with probability 1/n and value 0 otherwise; Y_n takes value n with probability 1/n and value 0 otherwise, so both are usually 0 but Y_n's rare event is much larger.
  • Moments diverge: E[X_n] = 1/n -> 0 and Var(X_n) -> 0, while E[Y_n] = 1 for every n and Var(Y_n) = n - 1 -> infinity.
  • Chebyshev works for X_n: because its variance shrinks fast enough, Chebyshev's inequality directly shows X_n converges to 0 in probability.
  • Chebyshev fails for Y_n: its variance diverges, so the Chebyshev bound becomes useless and cannot establish convergence.
  • Direct calculation still works: computing P(|Y_n| >= epsilon) directly from the distribution shows it equals 1/n, which goes to 0, proving Y_n also converges to 0 in probability.
  • Convergence in probability does not imply convergence in expectation: Y_n converges to 0 in probability while E[Y_n] stays at 1 forever, because a shrinking-probability event can carry an unboundedly large value.

Before you watch

  • Review the Chebyshev inequality and the definition of convergence in probability from the lecture on the weak law of large numbers.
  • Be comfortable computing expected value and variance for a discrete random variable with two possible values.

Check your understanding

  1. Why does the Chebyshev inequality prove convergence in probability for X_n but not for Y_n?
  2. How is P(|Y_n| >= epsilon) computed directly from Y_n's distribution, and why does this succeed where Chebyshev's inequality fails?
  3. Why can a sequence of random variables converge in probability to 0 while its expected value stays fixed at a nonzero number?
  4. What property of Y_n's rare event (its size versus its probability) causes the expected value to stay constant as n grows?

Vocabulary

sequence (noun)
An ordered list of numbers or random variables, one after another.
We study a sequence of random variables X_n and Y_n.
rare event (phrase)
An outcome that has a very small chance of happening.
Y_n's rare event carries a much larger value than X_n's.
diverge (verb)
To grow without limit, or to fail to settle to a fixed value.
The variance of Y_n diverges as n grows.
uninformative (adjective)
Giving no useful conclusion.
The Chebyshev bound becomes uninformative for Y_n.
unboundedly (adverb)
Without any upper limit.
A rare event can carry an unboundedly large value.
shrinking-probability (adjective)
Describing an event whose chance keeps getting smaller.
A shrinking-probability event can still affect the expected value.
Chebyshev inequality (noun)
A rule that bounds how likely a random variable is to be far from its mean, using its variance.
Chebyshev's inequality works directly for X_n because its variance shrinks fast.
weak law of large numbers (noun)
A rule stating that an average of many samples gets close to the true mean in probability.
This problem builds on the weak law of large numbers from the previous lecture.
expected value (noun)
The long-run average outcome of a random variable, weighted by probability.
The expected value of Y_n stays at 1 no matter how large n gets.
variance (noun)
A measure of how spread out a random variable's values are around its mean.
The variance of Y_n grows without bound as n increases.
probability distribution (noun)
A description of how likely each possible outcome of a random variable is.
We use Y_n's probability distribution directly to compute the exact chance.
bound (noun)
A limit that a value cannot go beyond.
Chebyshev's inequality gives an upper bound on the probability of a large deviation.
converge (verb)
To get closer and closer to a fixed value.
Both X_n and Y_n converge to 0 in probability.
convergence (noun)
The process of a sequence getting closer to a fixed value.
This problem tests convergence in probability versus convergence in expectation.
counterexample (noun)
A single case that proves a general claim is false.
Y_n is a counterexample showing convergence in probability does not imply convergence in expectation.
worked example (phrase)
A full problem solved step by step to show the method.
This lecture is a worked example comparing two sequences.
tail (noun)
The far, less likely part of a probability distribution, away from its center.
Y_n's tail holds a large value even though it is rarely reached.
magnitude (noun)
The size of a value, ignoring its direction or sign.
The magnitude of Y_n's rare outcome grows with n.
arbitrarily (adverb)
To any degree you choose, with no fixed limit.
We can make epsilon arbitrarily small and the probability still goes to 0.
inherently (adverb)
As a basic, built-in feature of something.
Convergence in probability is inherently a weaker statement than convergence in expectation.
establish (verb)
To show clearly that something is true.
Chebyshev's inequality cannot establish convergence for Y_n.
guarantee (verb)
To make certain that something will happen.
Convergence in probability does not guarantee convergence of the expected value.
fixed (adjective)
Not changing; staying at the same value.
E[Y_n] stays fixed at 1 for every n.
nonzero (adjective)
Not equal to zero.
The expected value stays at a nonzero number even as the probability shrinks.
carry (verb)
To have or hold a certain value or property.
The rare event can carry a very large value.
shrink (verb)
To become smaller over time.
The probability of the rare event shrinks as n grows.
roughly (adverb)
Approximately, not exactly.
For large n, X_n is roughly always equal to 0.
outcome (noun)
A possible result of a random process.
Each outcome of Y_n is either 0 or n.
offset (verb)
To balance out or cancel the effect of something else.
The growing size of the rare value offsets its shrinking probability, keeping the expected value fixed.
intuition (noun)
A natural feeling for how something works, before doing exact calculations.
This example is designed to challenge your intuition about convergence.

Chapters

From the YouTube description

MIT 6.041SC Probabilistic Systems Analysis and Applied Probability, Fall 2013
View the complete course: http://ocw.mit.edu/6-041SCF13
Instructor: Kuang Xu

License: Creative Commons BY-NC-SA
More information at http://ocw.mit.edu/terms
More courses at http://ocw.mit.edu

← Lecture 19: Weak Law of Large Numbers · Convergence in Probability and in the Mean, Part 2 →