Inferring a Parameter of the Uniform Distribution, Part 1

What this lecture covers

This is a worked review problem that reuses the Bayesian inference tools from Chapter 8 of the course. Romeo and Juliet, introduced earlier when Juliet's arrival delay was modeled as uniform between 0 and 1 hour, return with a twist: the upper bound of that uniform distribution, called theta, is now itself unknown and treated as a random variable, uniform between 0 and 1. The lecture derives the posterior distribution of theta after observing one delay, then extends the result to n independent observations.

By the end you should be able to set up a prior and a conditional (likelihood) distribution for a simple two-layer random model, apply Bayes' rule to get a posterior, and read off how each new data point narrows the range of plausible values for an unknown parameter. This sets up Part 2, which uses the posterior to build point estimators.

Key ideas

• Two layers of randomness: theta (the true upper bound of the delay distribution) is itself uniform between 0 and 1, and the observed delay x is uniform between 0 and theta, conditional on theta.
• Prior: the initial belief about theta before seeing data, here a flat uniform density on [0,1].
• Likelihood: the conditional PDF of x given theta, 1/theta for x between 0 and theta.
• Bayes' rule for the posterior: f(theta|x) = f(theta) * f(x|theta) / f(x), where the denominator is a normalizing constant that does not depend on theta.
• Posterior shape: for one observation, the posterior is proportional to 1/theta on the range [x, 1], so it rules out any theta smaller than the observed delay.
• Conditional independence across samples: given theta, multiple observed delays are treated as independent, so their joint likelihood is the product of individual 1/theta terms.
• Sufficient statistic: with n observations, only the maximum observed delay (x_bar) matters; the posterior becomes proportional to 1/theta^n on [x_bar, 1].
• More data sharpens the posterior: each additional observation both narrows the valid range for theta and makes the posterior more peaked near the sufficient statistic.

Walkthrough

Setting up the problem (1:00)

The lecture restates the classic Romeo-and-Juliet delay problem, now adding uncertainty about the uniform distribution's own upper bound theta. It defines theta as uniform on [0,1] and the delay x, conditional on theta, as uniform on [0, theta].

Writing down the prior and conditional distributions (2:01)

The PDF of theta is 1 on [0,1]. The conditional PDF of x given theta is 1/theta on [0, theta]. The lecture explains why the goal of inference is to combine this prior and the observed data into a posterior via Bayes' rule.

Deriving the posterior for one observation (4:01)

Applying Bayes' rule, the numerator is the product of the prior and the conditional PDF, valid only where theta is in [0,1] and x is in [0, theta]. The normalizing denominator is computed by integrating over theta from x to 1, giving |log x|. The resulting posterior is 1/(theta * |log x|) on [x, 1].

Interpreting the posterior (8:07)

The lecture explains what the shape means: observing a delay of, say, half an hour rules out any theta below half an hour, and values of theta closer to the observed x are more likely because a smaller theta concentrates more probability near that value.

Extending to multiple observations (12:16)

With n independent dates, each delay x_1 through x_n is conditionally independent given theta. Their joint conditional PDF is the product of individual 1/theta terms, giving 1/theta^n, valid only when theta is at least as large as the maximum observed delay, x_bar.

The posterior with n data points (16:34)

Applying Bayes' rule again, the posterior becomes proportional to 1/theta^n on [x_bar, 1]. The lecture notes the normalizing constant can be computed the same way as before but is less important than the shape, which becomes steeper and more concentrated as n grows.

Before you watch

• Be comfortable with the original Romeo-and-Juliet uniform-delay setup from earlier in the course.
• Review Bayes' rule for continuous random variables and how to compute a posterior PDF from a prior and a conditional PDF.
• Know how to integrate simple PDFs like 1/theta to find normalizing constants.

Check your understanding

1. Why is the posterior distribution restricted to the range [x, 1] rather than the full [0,1] prior range?
2. How does observing a delay of x rule out certain values of theta?
3. Why does conditional independence let you write the joint likelihood of n observations as a product?
4. Why does only the maximum observed delay matter when there are multiple observations?
5. What happens to the shape of the posterior as the number of observations n increases?