Lecture 8: Linearizability and Zookeeper

What this lecture covers

The lecture answers two questions: how do you prove that a sequence of reads and writes is (or isn't) linearizable, and why would a real system like Zookeeper deliberately give up that guarantee? It continues a discussion from the previous class and sits right before the course turns to building coordination services on top of replicated logs like Raft.

After watching, you should be able to take a history of client requests and responses and either construct a real-time-consistent total order that explains it, or find a cycle that proves no such order exists. You should also understand why replicated systems that let clients read from any replica risk returning stale data, and why Zookeeper accepts that risk for writes-only linearizability instead.

Key ideas

• Linearizability: a history is linearizable if there's a total order of operations that respects real time (an operation that finished before another started must come first) and in which every read returns the value of the most recent preceding write.
• Proof by ordering or by cycle: you can prove a history linearizable by exhibiting a valid total order, or prove it non-linearizable by finding a cycle in the "must come before" relation implied by the rules.
• Client-centric definition: linearizability only constrains what clients observe (request and response times); it says nothing about internal replica behavior.
• No stale reads: a linearizable system can never return a value older than the most recently completed write, which is why serving reads from lagging replicas breaks the guarantee.
• Retransmissions count from first send: when a client resends a dropped request, the real-time window for linearizability runs from the original send to the final response, not from the resend.
• Zookeeper's write guarantee: all writes (and operations that modify state) are linearizable with respect to each other, even though reads are not.
• FIFO client order: each client's own operations execute in the order it issued them, and successive reads from the same client never move backward in the replicated log, even across replicas.

Walkthrough

Finishing linearizability with worked examples (0:01)

The lecture works through several small histories of concurrent writes and reads on a single key, showing how to check linearizability. A history is linearizable if there's some total order of the operations, consistent with real time, where each read sees the value of the most recently preceding write. One example is shown to be linearizable by constructing such an order; a second, where two different clients each see the two concurrent writes in opposite orders, is shown to be impossible, because linearizability forbids different clients from observing different progressions of the data. The instructor stresses that this is a definition on observed histories, not on system designs: you can't certify a system linearizable in the abstract, only rule it out by finding a bad history.

Stale reads and retransmissions (22:32)

A simple history—write X=1, then write X=2 completes, then a read returns 1—is clearly not linearizable, because the only real-time-consistent order has the read violating the value rule. This is the formal argument behind the claim that linearizable systems can never serve stale data. The lecture then covers a subtler case: client retransmissions. If a client's request is dropped and it resends, and the server detects and replies to the duplicate with an old cached answer, that's still legal, because the real-time window for linearizability spans from the client's first send to its final received response, not from the resend.

Zookeeper's API motivation and consistency guarantees (36:44)

Zookeeper is introduced as a real-world, Raft-like (actually Zab-based) replicated coordination service, interesting for two reasons: it offers a general-purpose API rather than a library like Raft, and it tries to get more read throughput out of extra replicated servers. The lecture shows that in a plain leader-based replication scheme, adding servers doesn't help performance at all, because the leader is the bottleneck for every write and every read. Sending reads to followers instead of the leader would help, except that followers may lag behind the leader (not in the majority, missing a commit, or partitioned), so reading from them can return stale data—something linearizability forbids. Zookeeper resolves the tension by simply not promising linearizable reads: writes are linearizable, but reads can be stale, which is the "definitional wave of the wand" that lets it scale read throughput with the number of replicas.

FIFO client order and its consequences (52:33)

Given up-front freshness, Zookeeper instead guarantees FIFO client order: a given client's operations, reads and writes together, execute in the order it issued them, and reads never regress to an earlier point in the log even if the client switches replicas. This is implemented with ZXIDs, log entry numbers that each replica attaches to its responses; a client remembers the highest ZXID it has seen and includes it in later requests, and a replica must catch up to that point before answering.

Building on weak guarantees: the ready-file pattern and watches (1:07:05)

The lecture closes with an extended example from the paper: a master updates a multi-file configuration by first deleting a "ready" file, writing the individual configuration files, then recreating the ready file, so readers only trust the configuration when ready exists. FIFO client order alone isn't enough to prevent a reader from seeing a mix of old and new files, so Zookeeper adds watches: a client can ask to be notified if a file changes, and the guarantee is that the notification is delivered before any subsequent read result that reflects a state past the triggering event. This lets a reader detect, mid-read, that the configuration it's looking at just became invalid.

Before you watch

• Be comfortable with the definition of linearizability from the previous lecture, including real-time ordering and the "no stale reads" property.
• Know how Raft leaders and followers replicate a log, since Zookeeper's Zab layer is described as behaving almost identically.
• Recall Lab 3's put/get key-value interface, which the lecture references directly when discussing what linearizability requires.

Check your understanding

1. Given a history with two concurrent writes to the same key and two reads by different clients that see them in opposite orders, why can no valid total order exist?
2. Why does reading from a lagging follower in a Raft-like system risk violating linearizability, even if that follower is functioning correctly?
3. In what sense is Zookeeper "not linearizable," and what does it still guarantee about the order of writes?
4. How does the ZXID mechanism let a client preserve FIFO client order even after switching to a different replica?
5. Why isn't FIFO client order alone sufficient to make the ready-file configuration pattern safe, and what does the watch mechanism add?