Lecture 14: Poisson Process I

What this lecture covers

This lecture introduces the Poisson process, the continuous-time counterpart to the Bernoulli process covered earlier in the course. It answers the question of how to model arrivals that can happen at any instant rather than in fixed discrete slots, by treating the Poisson process as the limit of a Bernoulli process with more and more slots, each with a smaller and smaller success probability.

Building on the Bernoulli process material from the previous lecture, this session derives the Poisson PMF for the number of arrivals in an interval, shows that the time to the first arrival is exponentially distributed, and extends the memorylessness argument to continuous time. It closes by showing that merging two independent Poisson processes yields another Poisson process, setting up applications covered afterward.

Key ideas

• Time homogeneity: the distribution of the number of arrivals in an interval depends only on the interval's length, not its position on the time axis.
• Independence of disjoint intervals: the number of arrivals in non-overlapping time intervals are independent random variables.
• Arrival rate lambda: in a small interval of length delta, the probability of exactly one arrival is approximately lambda * delta, and the probability of two or more is negligible.
• Poisson PMF: the number of arrivals in an interval of length t follows P(k,t) = (lambda*t)^k / k! * e^(-lambda*t), obtained as the limit of the binomial PMF as the number of slots goes to infinity.
• Mean equals variance: both the expected number of arrivals and the variance of the count equal lambda * t.
• Exponential inter-arrival times: the time to the first arrival is exponentially distributed with parameter lambda, the continuous analog of the geometric distribution.
• Memorylessness: after any point in time, or after any arrival, the future of the process looks like a fresh Poisson process, independent of the past.
• Merging Poisson processes: two independent Poisson processes with rates lambda1 and lambda2 combine into a single Poisson process with rate lambda1 + lambda2, and a given arrival came from the first process with probability lambda1 / (lambda1 + lambda2).

Walkthrough

Reviewing the Bernoulli process (0:21)

The lecture opens by recalling the Bernoulli process from the previous session: a sequence of independent trials, each a success with probability p. The number of successes in n trials is binomial, the time to the first success is geometric, and the time to the k-th success is a sum of independent geometric random variables, known as the Pascal distribution. The language shifts from "successes" to "arrivals," which will carry over more naturally into continuous time.

The memoryless property in depth (4:58)

The lecture revisits memorylessness with a scenario: someone watches a sequence of Bernoulli trials and, based only on what they've seen so far, calls another person in to start watching. As long as the decision to call someone in does not depend on future outcomes, the trials that person then observes form a fresh Bernoulli process, independent of everything before. This causal argument is what justifies treating consecutive inter-arrival times as independent, identically distributed random variables.

Defining the Poisson process (9:10)

Motivated by recording continuous arrival times (such as customers entering a bank) instead of discretizing into slots, the lecture lays out the defining assumptions of the Poisson process: time homogeneity of arrival counts over intervals of equal length, and independence of arrival counts over disjoint intervals.

Arrivals in a small interval (14:24)

To pin down a specific formula, the lecture assumes that in a small interval of length delta, the probability of one arrival is approximately lambda * delta, the probability of zero arrivals is the remainder, and the probability of two or more arrivals is negligible as delta shrinks. This defines lambda as the arrival rate: the expected number of arrivals per unit time.

Deriving the Poisson PMF (18:45)

By splitting a long interval into many tiny sub-intervals, each behaving like a Bernoulli trial with success probability lambda * delta, the lecture treats the Poisson process as the limit of a Bernoulli process as the number of slots goes to infinity and the per-slot probability goes to zero. Taking this limit of the binomial PMF yields the Poisson PMF, with mean and variance both equal to lambda * t. A worked example computes the probability of zero and one email arriving in a half-hour window at a rate of five per hour.

Inter-arrival and k-th arrival times (32:54)

Using the same small-interval argument, the lecture derives the PDF for the time of the k-th arrival, and shows that setting k=1 recovers the exponential distribution — the continuous analog of the geometric distribution for the Bernoulli process's first-success time. It also notes a practical consequence: simulating a Poisson process is more efficient by directly sampling exponential inter-arrival times than by simulating tiny discrete slots.

Merging Poisson processes (42:16)

The lecture closes by showing that two independent Poisson processes merge into a single Poisson process whose rate is the sum of the individual rates, using the same small-interval, discard-higher-order-terms argument used throughout. It also derives that, given an arrival in the merged process, the probability it originated from a particular component process is proportional to that process's rate.

Before you watch

• Review the Bernoulli process: binomial counts, geometric inter-arrival times, and the Pascal distribution for the k-th arrival, from the previous lecture.
• Be comfortable with limits of the form (1 - c/n)^n -> e^(-c).
• Know the basic definition of a probability density function as giving probabilities over small intervals.

Check your understanding

1. Why does taking the limit of a Bernoulli process as the number of slots grows and the per-slot probability shrinks produce the Poisson PMF?
2. Why are the mean and variance of a Poisson-distributed count both equal to lambda * t?
3. How does the memorylessness argument used for the Bernoulli process carry over to justify independence of Poisson inter-arrival times?
4. If two independent Poisson processes have rates 2 and 3 arrivals per minute, what is the probability that a given merged arrival came from the first process?
5. Why is directly sampling exponential inter-arrival times a more efficient way to simulate a Poisson process than discretizing time into tiny slots?