Lecture 14: Midterm Review

What this lecture covers

This is a live question-and-answer review session before the CS149 midterm, not a normal lecture. The instructor asks students which topics from the first half of the course they want revisited, then works through each one on the board. The session ranges freely across the material covered so far: multicore architecture, synchronization primitives, cache coherence, memory consistency, data-parallel programming models, and CUDA.

Because it is student-driven, the order jumps between topics rather than building linearly. Watching it after finishing the earlier lectures on locks, cache coherence, and CUDA works as a way to check which ideas are still shaky, since the instructor repeatedly distinguishes what students need to reproduce on an exam from what is just background color.

Key ideas

• Mutual exclusion: only one thread can execute a protected section of code at a time; this is what a lock guarantees.
• Compare-and-swap (CAS): an atomic primitive that reads a memory location and, only if it still holds an expected old value, writes a new value; otherwise it leaves the value unchanged.
• Lock-free programming: instead of enforcing mutual exclusion, a thread does speculative work and uses CAS to check right before committing whether another thread got there first, retrying if so.
• MapReduce: a two-function programming model (map produces key-value pairs, a system-managed shuffle groups values by key, reduce aggregates each group) built for clusters where data cannot fit in memory on one machine.
• MSI protocol: a cache coherence protocol with Modified, Shared, and Invalid states per cache line, where any state change is broadcast to other caches.
• MESI protocol: MSI plus an Exclusive state, which lets a cache that is the sole owner of a clean line skip the broadcast when it later writes to it.
• Relaxed memory consistency: other processors may observe a thread's writes to different addresses in a different order than the thread issued them, even though the thread itself always sees its own operations in program order.
• Locality over scale-out: several real systems (a single-laptop benchmark, Amazon Prime Video's move off Lambda) got equivalent performance far more cheaply by keeping data local on fewer, bigger machines instead of distributing it across many small ones.

Walkthrough

Atomic compare-and-swap and lock-free Min (4:08)

The instructor reviews what mutual exclusion and atomic read-modify-write operations mean, then walks through implementing an atomic minimum using only compare-and-swap: read the current value, compute the minimum with a new value, and try to write it back only if the memory location has not changed since the read. If another thread updated it first, the CAS fails and the operation retries. This is contrasted with lock-based mutual exclusion, which would simply take a lock around the whole read-check-write sequence. The lock-free version allows two threads to run through the check concurrently as long as they don't ultimately conflict, at the cost of redoing wasted work on a failed CAS.

Lock-free tradeoffs and transactional memory (11:10)

Students ask about starvation and the overhead of compare-and-swap. The instructor explains that lock-free code has no built-in fairness guarantee, just like a naive lock, and that CAS is expensive because the cache coherence protocol must hold a line in a writable state across the compare-and-write instead of treating it as a plain read. He previews that later lectures extend this idea to transactional memory, where a function can read and write many variables and the system detects and resolves conflicts automatically, and notes that a correct lock-free linked list is hard enough to implement that it would not be a fair exam question.

MapReduce and the locality lesson (15:12)

In response to a question, the instructor reconstructs the MapReduce model from Google's original paper: map runs over every line of a distributed file and emits key-value pairs, the system shuffles pairs so all values for the same key end up together, and reduce runs once per key over its list of values. Because MapReduce round-trips through a distributed file system between phases, it was built assuming data too large to fit in memory. The instructor uses this to make a broader point: benchmarks later showed a single laptop streaming data from disk could outperform a large Spark or MapReduce cluster on data that fit locally, and Amazon's Prime Video team found a small number of large servers ten times more cost-efficient than a Lambda-based microservice architecture at the same performance, both illustrating that locality often beats scaling out when data fits on a machine.

Cache coherence: the MSI protocol in action (30:19)

The instructor recruits two student volunteers to physically play two caches and trace through the MSI protocol by hand on a shared variable X. They step through loading X into the Shared state, writing it (transitioning to Modified with no coherence traffic needed since no one else has it), and then a second cache requesting to read or write, which forces a flush of the dirty value back to memory and an invalidation of the first cache's copy. The exercise makes concrete that any state change in one cache must be communicated to the others, and that flushes move whole cache lines, not single values, because the protocol has no per-byte dirtiness tracking.

From MSI to MESI (48:33)

Building on the demo, the instructor introduces the Exclusive state used in modern (MESI) protocols. If a cache loads a value and no other cache reports having it, it can enter Exclusive rather than Shared, which means it can later write to that value by flipping straight to Modified without notifying anyone. This optimization targets the common read-then-write pattern and avoids an unnecessary broadcast that plain MSI would require.

Relaxed memory consistency (54:39)

The instructor separates cache coherence (which concerns operations on a single address) from memory consistency (which concerns the relative order of operations on different addresses as seen by other processors). A thread always observes its own operations in program order, but under relaxed consistency, another thread reading two variables that the first thread wrote may see the updates appear in a different order than they were issued. Exam-style questions on this topic tend to focus on read/write and write/write reordering across addresses.

CUDA thread block and warp scheduling (1:03:48)

The final segment ties CUDA back to the bulk-launch task system from earlier assignments: a CUDA launch specifies a number of thread blocks and a number of threads per block, and the hardware scheduler assigns blocks to cores based on available execution contexts and shared memory, filling gaps as blocks complete. The instructor then explains warps: the GPU groups threads (historically 32 consecutive thread IDs) into a warp that executes one instruction across all of them at once on a bank of ALUs. Because programmers write plain per-thread code rather than explicit vector instructions, Nvidia has flexibility to change the underlying SIMD width without requiring code changes, unlike CPU SIMD, where the compiler must target a fixed vector width such as an ISPC gang size.

Before you watch

• Review the earlier lectures on locks and atomic operations, cache coherence and the MSI protocol, memory consistency, MapReduce-style data-parallel thinking, and CUDA's thread block and warp model, since this session assumes all of them.
• Have the MSI/MESI state diagram in mind; the lecture works through it interactively rather than re-deriving it from scratch.

Check your understanding

1. Why does a lock-free minimum operation using compare-and-swap not guarantee mutual exclusion, and what happens when the CAS fails?
2. In MapReduce, what does the map phase produce, and what does the system do with that output before reduce runs?
3. Walk through what happens to a cache line's MSI state when one cache holds it in Modified and a second cache requests to read it.
4. What extra state does MESI add over MSI, and what specific case does it optimize?
5. Why can two processors observe a third processor's writes to different variables in different orders under relaxed consistency, even though that processor's own thread always sees them in program order?