Lecture 13: Fine-Grained Synchronization and Lock-Free Programming

What this lecture covers

This lecture asks how locks are actually implemented, why naive lock implementations create expensive coherence traffic, and how to make concurrent data structures (like a linked list) more concurrent than a single global lock allows. It starts with vocabulary for synchronization failures - deadlock, livelock, and starvation - then reconnects the earlier cache coherence lecture to lock implementation, building progressively better locks from atomic hardware instructions like test-and-set and compare-and-swap. It closes with an introduction to lock-free programming.

After watching, you should be able to explain the four conditions required for deadlock, distinguish it from livelock and starvation, trace the cache-coherence traffic generated by a spinlock built on test-and-set, explain why test-and-test-and-set and ticket locks reduce that traffic, use compare-and-swap to build a custom atomic operation, reason about where to place locks in a linked list to keep it both correct and concurrent, and describe the basic idea behind lock-free data structures.

Key ideas

• Deadlock: requires mutual exclusion, no preemption, hold-while-waiting (no relinquishing partial resources), and a circular chain of dependencies; breaking any one of these prevents it.
• Livelock: threads keep taking action (unlike deadlock) but make no real progress, for example when everyone repeatedly backs off and retries in lockstep.
• Starvation: some operations are indefinitely denied progress because other traffic always has priority, even though the system overall keeps doing useful work.
• Test-and-set lock: an atomic instruction that reads a memory location and sets it to one if it was zero; a simple lock loops calling test-and-set until it succeeds, but every attempt is a write, so it constantly invalidates the lock's cache line across contending cores.
• Test-and-test-and-set: spin on a plain read (which just takes cache hits in the shared state) and only attempt the atomic test-and-set once the value looks unlocked, which removes most of the coherence traffic during the critical section.
• Ticket lock: threads atomically increment a "next ticket" counter and spin reading a "now serving" counter; unlocking only requires one write, and the lock is fair (first-come, first-served).
• Compare-and-swap (CAS): an atomic instruction that writes a new value into memory only if the current value still matches an expected old value, otherwise leaves memory unchanged and reports the actual value; it is general enough to build custom atomic operations like atomic-min and to build locks themselves.
• Hand-over-hand locking: to keep a linked list correct while allowing concurrent access to different parts of it, a thread must hold locks on both the current and previous node before modifying links, acquiring the next lock before releasing the previous one.
• Lock-free programming: instead of blocking, a thread speculatively performs work and, at the last step, uses an atomic operation like CAS to check that nothing changed; if something changed, it retries, avoiding the risk that a thread holding a lock gets descheduled while others wait.

Walkthrough

Deadlock, livelock, and starvation (0:05)

Using intersection and traffic examples, the lecture defines the four conditions needed for deadlock - mutual exclusion, no preemption, holding resources while waiting for more, and a circular dependency chain - and shows that removing any one of them prevents it. It contrasts this with livelock, where threads keep taking visible action but make no real progress (for example, everyone repeatedly yielding and then all trying to move at once), and starvation, where some operations are permanently denied service even though the system is otherwise making progress, illustrated with a yield-sign traffic example.

Reviewing cache coherence through a lock's eyes (9:10)

Before building locks, the lecture revisits the MSI-style coherence protocol from the prior lecture, working through what each cache does as one processor loads, re-reads, and writes a variable while others snoop the bus. This sets up the idea that a lock variable is just an ordinary cached memory location, so understanding a lock's performance requires understanding the coherence traffic its accesses generate.

Building a lock from test-and-set (17:20)

The lecture introduces test-and-set, an atomic instruction that reads a memory location and sets it to one if it was zero, returning the old value. A lock built from it loops calling test-and-set until it returns zero (meaning the caller just acquired the lock); unlock simply writes zero. Because every attempt is a write from a coherence perspective, even failed attempts invalidate the lock's cache line on every contending processor, so the line bounces between processors constantly while threads spin - and the cost of acquiring the lock rises with the number of contending cores.

Reducing coherence traffic: test-and-test-and-set and ticket locks (21:22)

To cut this traffic, the lecture refines the lock to spin on an ordinary read first (which just takes cache hits in shared state across all waiting threads) and only attempt the atomic test-and-set once the lock looks free. This confines write traffic to the moment of release, though it still produces a burst of failed attempts from every waiting thread at once. The ticket lock improves further: each thread atomically increments a shared "next ticket" counter to get its position, then spins reading a separate "now serving" counter; releasing the lock is a single write that increments "now serving." This requires an atomic increment rather than test-and-set, needs only one write per release, and is fair because threads are served in the order they arrived.

Atomic operations and compare-and-swap (35:33)

The lecture generalizes to a library of atomic hardware operations (as found in CUDA, for example), focusing on compare-and-swap (CAS): it writes a new value to memory only if the current value still equals an expected old value, and always returns what was actually in memory. Using CAS, the lecture builds an atomic-min operation: read the current value, compute the candidate minimum, then CAS it in only if the value hasn't changed since the read - retrying if another thread updated it first. The same pattern builds lock and unlock themselves, and the lecture notes that real lock implementations on relaxed-memory-consistency hardware need a memory fence around the critical section to avoid reordering that would make the lock ineffective.

Fine-grained locking on a linked list (52:50)

Working through insert and delete on a sorted linked list, the lecture shows how running them unsynchronized can lose inserted nodes, corrupt the list during simultaneous insert-and-delete, or double-free memory. A single lock around the whole list is correct but serializes all access, eliminating concurrency benefits. The lecture develops hand-over-hand locking instead: to delete or modify a node, a thread must hold locks on both that node and its predecessor, acquiring the next node's lock before releasing the previous one, which guarantees at least one lock is always held during traversal and prevents another thread from invalidating the path.

Lock-free data structures (1:10:11)

The lecture closes by introducing lock-free (non-blocking) programming: instead of blocking on a lock, a thread speculatively performs its work and, at the last step, uses an atomic check like compare-and-swap to confirm nothing else changed, retrying if it did. This avoids a key weakness of blocking locks - a thread holding a lock can be descheduled, stalling every other waiting thread - which matters especially in systems with many more threads than cores, such as servers handling concurrent I/O. A simple two-thread queue (one thread only advancing the head, the other only the tail) is given as a case that is inherently thread-safe without any locking at all.

Before you watch

• Watch the previous lecture on cache coherence, since this lecture reuses its state-transition reasoning to explain why naive locks are expensive.
• Be comfortable with basic multithreaded programming concepts (critical sections, mutual exclusion) from earlier in the course.
• Familiarity with pointer-based data structures (linked lists) will help with the fine-grained locking discussion.

Check your understanding

1. Name the four conditions required for deadlock, and describe a real-world scenario where removing just one of them prevents it.
2. Why does a simple test-and-set spinlock generate a burst of bus traffic even from threads that fail to acquire the lock, and how does test-and-test-and-set reduce it?
3. Explain how a ticket lock guarantees fairness, and what atomic hardware operation it depends on.
4. How would you implement an atomic-min operation using only compare-and-swap, and why must it retry rather than simply write its computed result?
5. In the hand-over-hand locking scheme for a linked list, why must a thread acquire the lock on the next node before releasing the lock on the current one?