Recitation: Inferring a Discrete Variable from a Continuous Measurement

What this lecture covers

This recitation problem practices the mixed version of Bayes' rule: inferring a discrete random variable X from a continuous measurement Z. X takes values 1, -1, or 0 with given probabilities, noise Y follows a two-sided exponential density, and the observed quantity is Z = X + Y. The task is to find the conditional probability that X = 1 given an observed value of Z.

It follows the lecture on continuous Bayes' rule, applying the formula for a discrete unknown measured through a continuous, noisy channel, and finishes by checking the result against several limiting cases to build confidence that the formula is correct.

Key ideas

• Mixed Bayes' rule: for a discrete X and continuous Z, the posterior PMF of X given Z = z is the prior PMF of X times the conditional density of Z given X, divided by the total probability, obtained via the law of total probability over the discrete values of X.
• Shifting a density to condition on X: because Z = X + Y, the conditional density of Z given X = 1 is just the noise density shifted by 1, and given X = -1 it is shifted by -1.
• Sanity-checking with limiting cases: plugging in extreme values of the parameters (P near 0 or 1, lambda near 0 or infinity) into the derived formula should match intuitive expectations, which confirms the algebra.
• Uninformative measurements: when the noise is very spread out (lambda near 0), observing Z does not change the belief about X, so the posterior equals the prior.
• Highly informative measurements: when the noise is nearly zero (lambda to infinity), Z essentially equals X, so the sign of the observed z determines X with near certainty.

Walkthrough

Setting up the model (0:00)

X is discrete with probability P of being 1, 1-P of being -1, and 0 otherwise. Y is a continuous two-sided exponential noise term with rate lambda. Z = X + Y is the observed, continuous measurement, and the goal is P(X=1 | Z=z).

Applying the mixed Bayes' rule (1:02)

The formula multiplies the prior PMF of X by the conditional density of Z given X, then divides by the total probability of Z=z, expanded using the law of total probability over X = 1 and X = -1. Since Z given X=1 is just the noise density shifted right by 1, and Z given X=-1 is the noise density shifted left by 1, both conditional densities can be written explicitly and substituted in.

Simplifying the expression (5:15)

After substituting the shifted exponential densities, the common factors cancel, leaving a formula for the posterior probability in terms of P, lambda, and z, involving a ratio of exponential terms.

Checking limiting cases (7:21)

As P goes to 0, the posterior probability correctly goes to 0, since X can never be 1. As P goes to 1, the posterior goes to 1. As lambda goes to 0 (very spread-out noise), the posterior simplifies back to P, meaning the measurement adds no information. As lambda goes to infinity (nearly noiseless), the posterior approaches 1 when z is positive and 0 when z is negative, since Z essentially reveals X directly.

Before you watch

• Review the discrete-and-continuous mixed form of Bayes' rule from the lecture on continuous Bayes' rule and derived distributions.
• Be familiar with the two-sided (Laplace-style) exponential density and how shifting a random variable shifts its density.
• Knowing the law of total probability for a discrete variable helps follow how the denominator is built.

Check your understanding

1. Why is the conditional density of Z given X=1 just the noise density shifted by 1?
2. What role does the law of total probability play in the denominator of the mixed Bayes' rule formula?
3. Why does the posterior probability equal the prior P when lambda goes to 0?
4. Why does observing the sign of z determine X almost exactly when lambda goes to infinity?