Seyed Masoud Hosseini · Overview · Study log · Weekly summaries · Ideas · Search · Transcript · RSS feed

Probability · Lecture 49 of 76 · 22:58

A Coin with Random Bias

A Coin with Random Bias on YouTube

Study guide

What this lecture covers

This is a worked recitation problem that layers randomness on top of randomness: a coin is flipped n times, but the coin's own bias is a random variable Q with known mean mu and variance sigma squared. The problem asks for the expectation of individual flips, the covariance between flips, and the variance of the total number of heads, computed two different ways.

It brings together several tools from the main lecture on iterated expectations: the law of iterated expectations, the covariance formula, and the law of total variance, applied to a hierarchy where the randomness of each coin flip depends on the randomness of the bias above it. After watching, you should be able to recognize when conditioning on a hidden parameter simplifies a layered random experiment, and compute covariance between dependent Bernoulli trials that share a common random bias.

Key ideas

  • Setup: Xi is 1 if the i-th flip is heads, 0 otherwise; conditional on Q, all flips are independent; E[Q] = mu, Var(Q) = sigma^2 > 0.
  • Conditional expectation of a flip: E[Xi|Q] = Q, so by the law of iterated expectations E[Xi] = E[Q] = mu, and E[X] = n*mu for the total number of heads X.
  • Covariance between different flips is not zero: for i != j, Cov(Xi, Xj) = E[Q^2] - mu^2 = Var(Q) = sigma^2, because sharing an unknown bias correlates the flips even though they are conditionally independent given Q.
  • Correlated implies not independent: since sigma squared is positive, Xi and Xj are correlated, and correlated random variables cannot be independent (though the converse - uncorrelated implies independent - is not generally true).
  • Variance of a single flip: Var(Xi) = mu - mu^2.
  • Two ways to compute Var(X): the law of total variance (conditioning on Q) and the covariance-sum formula (summing variances plus all pairwise covariances) give the same result, n*(mu - mu^2) + n*(n-1)*sigma^2.
  • Rule of thumb for layered randomness: when one layer of randomness (coin flips) depends on another (the bias), condition on the upper layer to simplify calculations at the lower layer.

Walkthrough

Setting up notation and conditional independence (0:00)

The problem defines the random bias Q, the Bernoulli indicators Xi for each flip, and states the key assumption that flips are independent conditional on Q, which will be used throughout.

Expectation of a single flip and of the total (1:01)

Conditioning on Q gives E[Xi|Q] = Q, so by the law of iterated expectations E[Xi] = mu; summing over n flips using linearity of expectation gives E[X] = n*mu.

Covariance between flips (5:06)

For i != j, the problem computes E[Xi*Xj] by conditioning on Q and using conditional independence, arriving at Cov(Xi, Xj) = sigma^2, showing the flips are correlated because they share the same unknown bias. It also computes Var(Xi) = mu - mu^2 for the i = j case, using the fact that a Bernoulli variable squared equals itself.

Variance of the total via the law of total variance (12:22)

Conditioning on Q, the problem computes E[Var(X|Q)] = n*(mu - sigma^2 - mu^2) and Var(E[X|Q]) = n^2 * sigma^2, combining them into Var(X) = n*(n-1)*sigma^2 + n*(mu - mu^2).

Cross-check via the covariance-sum formula (18:45)

The problem recomputes Var(X) by summing the n individual variances and the n^2 - n pairwise covariances directly, confirming it matches the law-of-total-variance result, and closes with a rule of thumb for choosing what to condition on in layered random experiments.

Before you watch

  • Watch the main lecture on iterated expectations for the law of iterated expectations, the covariance formula, and the law of total variance.
  • Know the mean and variance of a Bernoulli random variable.

Check your understanding

  1. Why are Xi and Xj correlated even though they are conditionally independent given Q?
  2. Derive Cov(Xi, Xj) = sigma^2 for i != j.
  3. Walk through both methods used to compute Var(X) and confirm they agree.
  4. What rule of thumb does the problem suggest for choosing what to condition on in a layered random experiment?

Vocabulary

layered randomness (phrase)
A situation where one random process depends on the outcome of another random process above it.
The coin's bias being random itself creates layered randomness.
hidden parameter (phrase)
An unknown value that controls a process but is not directly observed.
Q is a hidden parameter controlling the flips.
conditional independence (phrase)
Independence that only holds once a certain value is known.
The flips are independent only conditional on Q.
covariance (noun)
A number showing whether two variables tend to move together.
Covariance between flips comes from sharing the same hidden bias.
correlated (adjective)
Linked so that knowing one value gives information about another.
The flips are correlated because they share a random bias.
uncorrelated (adjective)
Having zero covariance, showing no linear link between two variables.
Uncorrelated variables can still fail to be independent.
cross-check (verb)
To confirm a result using a different method.
We cross-check the variance using the covariance-sum formula.
rule of thumb (phrase)
A simple, practical guideline based on experience rather than exact proof.
The rule of thumb is to condition on the higher layer of randomness first.
pairwise covariance (phrase)
The covariance calculated separately for each pair of variables.
The variance formula sums up all the pairwise covariances.
indicator variable (phrase)
A variable that equals 1 if an event happens and 0 otherwise.
Xi is an indicator variable for whether the i-th flip is heads.
converse (noun)
The reversed version of a statement, swapping its condition and conclusion.
The converse of this rule, uncorrelated implies independent, is not generally true.
notation (noun)
A system of symbols used to write mathematical ideas.
The problem sets up notation for the bias Q and the flip indicators Xi.
layer (noun)
One level in a structure that has several levels stacked together.
Coin flips form one layer, and the random bias forms the layer above it.
Bernoulli trial (phrase)
A single random experiment with exactly two possible outcomes.
Each coin flip is a Bernoulli trial with success probability Q.
share (verb)
To have something in common with another thing.
The flips share the same unknown bias Q.
arrive at (phrasal verb)
To reach a result after a process of reasoning or calculation.
The problem arrives at Cov(Xi, Xj) = sigma^2.
confirm (verb)
To show that something is true, often by checking it a second way.
The cross-check confirms the two methods give the same variance.
squared (adjective)
Multiplied by itself.
A Bernoulli variable squared equals itself.
law of total variance (phrase)
A rule for splitting overall variance into two parts based on conditioning.
The law of total variance gives one way to compute Var(X).
law of iterated expectations (phrase)
The rule that averaging a conditional expectation over all cases gives the overall expectation.
The law of iterated expectations gives E[Xi] = mu.
dependent (adjective)
Influenced by, or connected to, another variable.
The flips are dependent because they share a common random bias.
trial (noun)
One repetition of a random experiment.
The coin is flipped n times, once per trial.
combine (verb)
To join separate parts together into one result.
We combine the variance and covariance terms into one formula.
positive (adjective)
Greater than zero.
Since sigma squared is positive, Xi and Xj are correlated.
define (verb)
To state exactly what something means or how it is set up.
The problem defines the random bias Q and the indicators Xi.

Chapters

From the YouTube description

MIT 6.041SC Probabilistic Systems Analysis and Applied Probability, Fall 2013
View the complete course: http://ocw.mit.edu/6-041SCF13
Instructor: Jimmy Li

License: Creative Commons BY-NC-SA
More information at http://ocw.mit.edu/terms
More courses at http://ocw.mit.edu

← A Random Number of Coin Flips · 13. Bernoulli Process →