Inferring a Continuous Random Variable from a Discrete Measurement

What this lecture covers

This lecture works a single example problem: given a prior belief about the unknown bias Q of a coin, expressed as a continuous PDF, how do you update that belief after observing one flip, which is a discrete outcome (heads or tails)? It sits within the unit on Bayes' rule, extending the discrete-discrete version to the mixed case where the unknown is continuous but the observation is discrete.

After watching, you should be able to set up and solve this continuous-discrete version of Bayes' rule: write the numerator as a prior PDF times a conditional PMF, expand the denominator with the law of total probability as an integral, and interpret the resulting posterior PDF.

Key ideas

• Continuous prior, discrete observation: Q (the coin's bias) is modeled with a PDF, while the flip outcome X is a discrete Bernoulli random variable, so inference between them needs a mixed form of Bayes' rule.
• Prior on Q: the example prior is f(q) = 6q(1-q) for q between 0 and 1, a symmetric curve peaking at q = 1/2.
• Model of X given Q: given the bias q, X is Bernoulli with P(X=1|Q=q) = q and P(X=0|Q=q) = 1-q.
• Continuous-discrete Bayes' rule: the posterior PDF of Q given X is the prior PDF times the conditional PMF, divided by the PMF of X.
• Law of total probability for the denominator: the PMF of X is found by integrating the numerator expression over all possible values of Q.
• Posterior after heads: observing X = 1 shifts the posterior to 12q^2(1-q), with its peak moving up to q = 2/3.
• Posterior after tails: observing X = 0 shifts the posterior to 12q(1-q)^2, symmetric to the heads case, with its peak moving down.
• The beta distribution: the prior and both posteriors all belong to the beta family, commonly used to model probabilities and other quantities bounded between 0 and 1.

Walkthrough

Setting up the coin-bias problem (0:06)

The lecture introduces Q as the unknown bias of a coin, with prior PDF 6q(1-q) on [0, 1]. It explains that a flip of the coin is modeled as a Bernoulli random variable X, equal to 1 for heads and 0 for tails, with P(X=1|Q=q) = q. The conditional PMF of X given Q is written compactly as q when x=1 and 1-q when x=0.

Writing Bayes' rule for a continuous unknown and discrete data (2:06)

The goal is the conditional PDF of Q given X. Because Q is continuous and X is discrete, the numerator combines a PDF (the prior on Q) with a conditional PMF (the model of X given Q), and the denominator, the PMF of X, is obtained via the law of total probability by integrating that same product over all values of Q.

Solving for the case of observing heads (4:10)

For X = 1, the numerator becomes 6q(1-q) * q, and the denominator is that expression integrated over q from 0 to 1. Carrying out the calculus gives a denominator of 1/2, so the posterior simplifies to 12q^2(1-q). Plotted, this posterior peaks at q = 2/3: seeing heads is evidence the coin is biased toward heads, so belief in Q shifts above 1/2.

Solving for the case of observing tails (8:13)

Repeating the same steps for X = 0 swaps the roles of q and 1-q in the conditional PMF, leading to a posterior of 12q(1-q)^2. This mirrors the heads case and peaks at a low value of q, reflecting that observing tails is evidence the coin favors tails.

Recognizing the beta distribution (10:18)

The lecture points out that the prior and both posteriors are symmetric versions of each other and all belong to the beta family of distributions, which is commonly used to model a bias or any quantity restricted to [0, 1]. It closes by noting that this example is good practice for applying the continuous-discrete version of Bayes' rule, and recommends checking that a computed answer makes intuitive sense.

Before you watch

• Be comfortable with the discrete-discrete and continuous-continuous versions of Bayes' rule before working through this mixed case.
• Know the definition of a Bernoulli random variable and how to work with conditional PMFs.
• Be ready to do a short integral by hand, since the derivation depends on integrating a polynomial in Q.

Check your understanding

1. Why does inferring Q from a coin flip require a version of Bayes' rule that mixes a PDF and a PMF, rather than the purely discrete or purely continuous versions?
2. How is the denominator in this version of Bayes' rule computed, and what principle justifies that computation?
3. Why does the posterior after observing heads peak at a higher value of Q than the prior did?
4. What is the relationship between the two posterior distributions obtained from observing heads versus tails?
5. What family of distributions do the prior and posteriors in this problem belong to, and why is that family a natural choice here?