Lecture 12: Distributed Transactions

What this lecture covers

When data is sharded across many servers, some operations need to read or write records on more than one server at once, such as transferring money between accounts stored on different machines. The lecture asks what it means for such an operation to run correctly despite concurrency and failures, and how to build a system that guarantees it.

It works through the two implementation pieces that make this possible: concurrency control, which keeps concurrent transactions from seeing each other's intermediate state, and atomic commit, which makes sure a transaction's effects either all happen or none do, even when individual servers or the network fail. After watching, you should be able to explain serializability precisely, trace two-phase locking on a simple example, and walk through two-phase commit's message flow and its failure-handling rules.

Key ideas

• ACID: a database correctness standard; the lecture focuses on atomicity (all-or-nothing despite failure), isolation (transactions can't observe each other's intermediate state), and durability (committed data survives crashes).
• Serializable execution: a concurrent execution of transactions is correct if its result matches some one-at-a-time (serial) ordering of the same transactions.
• Pessimistic vs. optimistic concurrency control: pessimistic (locking) avoids conflicts by waiting; optimistic lets transactions proceed and checks for conflicts only at the end, aborting if one occurred.
• Two-phase locking: a transaction acquires locks as it accesses records and holds every lock until it commits or aborts; releasing a lock early can let another transaction see an uncommitted or later-undone value.
• Deadlock: two-phase locking can deadlock when transactions acquire the same locks in different orders; the database detects and breaks this by aborting one of the transactions.
• Atomic commit: a protocol that lets several servers, each responsible for part of a transaction, agree to either all apply their part or all abandon it.
• Two-phase commit (2PC): a transaction coordinator collects yes/no votes from participants via a prepare message, then sends commit only if every participant voted yes, otherwise it sends abort.
• Blocking: a participant that has voted yes but not yet heard the outcome must wait indefinitely rather than guess, because the coordinator may already have told another participant to commit.

Walkthrough

Why distributed transactions are needed (0:00)

The lecture opens with the motivation: sharding data across servers for load and space reasons means some operations, like a bank transfer, must touch records on more than one machine. Transactions are introduced as the abstraction that lets a programmer group several reads and writes into a single unit that appears atomic and isolated from other activity, illustrated with a transfer transaction and a concurrent audit transaction reading the same two balances.

Defining correctness with ACID and serializability (6:06)

ACID is introduced, with isolation singled out as the property that matters most for this lecture: transactions cannot see each other's intermediate updates. The formal definition of serializability follows: a concurrent execution is correct if there exists some serial (one-at-a-time) ordering of the same transactions that produces the same results. Applying this to the two-account example yields exactly two legal outcomes, and the lecture checks several possible interleavings against them to show which are legal and which are not.

Concurrency control: two-phase locking (20:28)

Pessimistic and optimistic concurrency control are contrasted, with the tradeoff being locking overhead versus abort frequency under conflicts. The lecture then works through two-phase locking in detail: acquire a lock before touching any record, and never release a lock until the transaction has committed or aborted. Several scenarios show why early release breaks correctness, including a case where a transaction reads a value written by another transaction that later aborts, producing a result that corresponds to no valid database state. Deadlock is introduced as an unavoidable side effect of this rule, handled by aborting one of the deadlocked transactions.

Two-phase commit protocol (37:55)

With locking established, the lecture turns to distributed atomic commit. A transaction coordinator drives the transaction and sends operations to participants holding the data; each message and piece of state is tagged with a transaction ID. The protocol itself: the coordinator sends prepare messages, each participant checks whether it can complete its part and replies yes or no, and the coordinator commits only if every participant said yes, otherwise it sends abort. Participants release their locks only after receiving the final commit or abort.

Coping with crashes and lost messages (47:07)

This is the core of the lecture: working through every point at which a participant or the coordinator could crash or a message could be lost. A participant that crashes before voting yes can unilaterally abort on recovery. One that voted yes must persist its transaction state to disk before replying, so it can honor a later commit even after a crash. A participant that has voted yes and is waiting for the outcome cannot time out and abort or commit on its own, because the coordinator may already have told another participant to commit; it must block until it hears from the coordinator. The coordinator similarly must log its commit/abort decision before sending any commit messages, so it can resend them after a crash.

Why two-phase commit is slow and unavailable, and how it combines with Raft (1:06:41)

The lecture closes by weighing two-phase commit's costs: multiple message rounds and mandatory disk writes before replying, both of which hold locks and slow other transactions. It is contrasted with Raft: Raft needs only a majority of replicas doing the same work, so it stays available despite failures, while two-phase commit needs every participant, each doing different work, to respond, so any failure can force the whole system to block. The lecture ends by describing how replicating the coordinator and each participant with Raft combines the two protocols to get both high availability and atomic multi-shard commits, which is the structure used in the course's sharded-database lab and in Google's Spanner, covered next.

Before you watch

• Be comfortable with the idea of locking from earlier labs, since two-phase locking builds directly on acquire/release lock semantics.
• Review Raft's leader-and-majority replication model, since the lecture explicitly contrasts it with two-phase commit near the end.
• Know what a database transaction (begin/commit) is meant to guarantee before the lecture formalizes it with ACID.

Check your understanding

1. Why must a transaction hold all its locks until it commits or aborts, rather than releasing them as soon as it finishes using a record?
2. Given the two-account transfer and audit transactions in the lecture, what are the only two legal final outputs, and why?
3. Why can a participant that voted yes in two-phase commit not simply abort after a long timeout waiting for the coordinator's decision?
4. What does a participant need to write to disk, and when, in order to survive a crash without breaking atomicity?
5. Why does replicating the coordinator and each participant with Raft improve availability without changing what two-phase commit guarantees?