Lecture 10: Continuous Bayes' Rule; Derived Distributions

What this lecture covers

The lecture has two parts. First, it extends Bayes' rule to every combination of discrete and continuous random variables: both discrete, both continuous, a discrete unknown measured by a continuous instrument, and a continuous unknown measured by a discrete instrument. Second, it introduces derived distributions, a systematic method for finding the distribution of a function of a random variable, such as Y = g(X), from the distribution of X.

It comes right after the lecture on multiple continuous random variables and prepares for the recitation and later problems where you compute distributions of transformed random variables, plus the inference chapter that returns to Bayesian estimation in depth near the end of the course.

Key ideas

• Bayes' rule generalizes across variable types: the formula "posterior proportional to prior times likelihood" holds whether the unknown and the measurement are discrete, continuous, or mixed; only whether you use a PMF or PDF changes.
• Inference as inverting conditioning: given a model of how a measurement Y depends on an unknown X, Bayes' rule produces the distribution of X given an observed Y.
• Mixed cases require care with notation: when one variable is discrete and the other continuous, the multiplication rule for probabilities becomes a formula mixing a PMF and a PDF, derived by working with small intervals and canceling their lengths.
• Derived distributions use a two-step recipe: write the CDF of Y = g(X) by translating the event into a statement about X, then differentiate to get the PDF of Y.
• Monotonic direction matters: for a decreasing function like T = 200/V, inequalities flip when translating from Y's event to X's event.
• Linear transformations: for Y = aX + b, the density of X is stretched by |a|, rescaled to keep area equal to 1, and shifted by b.

Walkthrough

Recap and the case for Bayesian inference (1:00)

The lecture briefly reviews how discrete and continuous concepts (joint distributions, conditional distributions) run in parallel, with PDFs playing the role of PMFs. It frames inference as the core use of probability: an unknown quantity X is measured through some noisy process producing an observable Y, and Bayes' rule lets you go from a model of Y given X to the distribution of X given the observed Y.

Bayes' rule for discrete and for continuous variables (6:13)

For two discrete variables, Bayes' rule takes its familiar form using PMFs. For two continuous variables, such as a noisy analog measurement of a current, the same formula holds with PDFs replacing PMFs, since the joint density is still the product of a marginal density and a conditional density.

Mixed discrete-continuous inference (9:18)

When X is discrete (for example a transmitted bit) and Y is a continuous noisy measurement, the lecture derives the mixed version of Bayes' rule by working with small intervals of Y and canceling their length, arriving at a formula involving the PMF of X and the conditional PDF of Y given X. The reverse case, a continuous unknown measured through a discrete count (such as photon counts from a light source), swaps the roles: the formula uses a PDF for the unknown and a PMF for the conditioning on the measurement.

Introducing derived distributions (20:36)

The lecture turns to finding the distribution of Y = g(X) from the distribution of X. It first notes that if you only need the expected value of g(X), you can compute it directly without ever finding the full distribution of Y. When you do need the distribution, in the discrete case you sum the probabilities of all x values that map to a given y; in the continuous case that direct approach gives probability zero for any single point, so instead you work with the CDF of Y, translating the event Y <= y into an equivalent event about X, then differentiate to get the density.

Worked examples of the two-step method (26:43)

The lecture applies the cookbook procedure to two examples. First, X is uniform on [0,2] and Y = X^3; translating Y <= y into X <= y^(1/3) and differentiating gives a non-uniform density for Y on [0,8], showing that a function of a uniform variable is not generally uniform. Second, T = 200/V for a uniform cruise-control speed V on [30,60] illustrates a decreasing function, where translating the event T <= t requires reversing the inequality on V, and produces a different density shape than the increasing case.

Linear functions of a random variable (38:07)

For Y = aX + b, the lecture first argues visually that the density of X gets stretched horizontally by a factor of a, rescaled vertically to preserve total area, and shifted by b. It then re-derives this with the CDF-and-derivative method, showing that for negative a the scaling factor must be |a| to keep the density non-negative, since dividing an inequality by a negative number flips its direction.

Before you watch

• Review the joint, marginal, and conditional densities from the lecture on multiple continuous random variables.
• Be comfortable with Bayes' rule and the multiplication rule for discrete random variables, since this lecture extends both to continuous and mixed cases.
• A working knowledge of differentiating composite functions (the chain rule) is needed for the derived-distribution examples.

Check your understanding

1. Why does Bayes' rule keep the same overall structure whether the variables involved are discrete, continuous, or mixed?
2. What are the two steps of the cookbook procedure for finding the distribution of Y = g(X)?
3. Why does the direction of an inequality flip when working with a decreasing function like T = 200/V?
4. Why must the density of Y = aX + b be scaled by |a| rather than a when a is negative?
5. Why can you skip finding the full distribution of g(X) if all you need is its expected value?