Lecture 11: Cache Coherence

What this lecture covers

The lecture has two parts. It first finishes the course's discussion of Spark: how the runtime fuses chains of RDD transformations when dependencies are narrow, how lineage gives fault tolerance without storing intermediate data on disk, and why scaling out to a cluster only pays off once data no longer fits on a single machine. It then turns to the lecture's main topic, cache coherence: why caching shared data across multiple private caches breaks the intuitive contract that a read should return the last value written, what invariants a correct memory system must satisfy, and how snooping-based protocols enforce them over a bus.

By the end you should be able to explain when Spark can fuse transformations without cross-node communication, why distributed systems only help once a dataset outgrows a single server's memory, why multiple private caches can silently disagree about a variable's value, and what the single-writer/multiple-reader and data-value invariants require of a coherent system. The lecture introduces the MSI protocol's states and bus transactions but does not yet walk through the full protocol, which continues in the next lecture.

Key ideas

• Narrow vs. wide dependencies: Spark can fuse a chain of RDD transformations into one pass with no extra memory traffic when each output partition depends on only one input partition; operations like group by key need data from every partition and force communication.
• Lineage-based fault tolerance: Spark keeps a log of the deterministic transformations used to build each RDD instead of replicating intermediate data, and recovers from a node failure by replaying that log from the replicated source data.
• Scale up vs. scale out: scale up means sharing memory across cores on one machine; scale out means separate machines communicating over a network. Distributed systems only outperform a single machine once the dataset no longer fits in that machine's memory.
• Three C's of cache misses: cold misses (first access), capacity misses (cache too small to hold everything needed), and conflict misses (a line's possible locations are limited by set associativity).
• Write-back vs. write-through, allocate vs. no-allocate: write-back caches only update memory when a dirty line is evicted; write-through caches update memory on every write; write-allocate fetches a line into the cache on a write miss before updating it.
• The coherence problem: once shared data is cached in multiple private caches, one processor's cache can hold a value that no longer matches what another processor last wrote, breaking the expectation that a load returns the last stored value.
• Single-writer/multiple-reader and data-value invariants: at any time, a cache line is either writable by exactly one processor or readable by any number of processors, and every reader in a read-only period must see the value from the most recent read-write period.
• Snooping over a bus: a bus provides both serialization (one transaction at a time) and broadcast (every cache sees every transaction), which a coherence protocol like MSI (modified, shared, invalid) uses to track and enforce exclusive write access.

Walkthrough

Finishing Spark: fusing narrow dependencies (0:05)

The lecture resumes the Spark discussion, reviewing RDDs as read-only, ordered collections built by applying deterministic transformations (filter, map) and consumed by actions. It revisits how the Spark runtime detects narrow dependencies - where each output partition depends on only one input partition - and fuses a whole chain of transformations into a single pass that reads one record at a time without extra memory traffic. Operations like group by key or a join across differently partitioned RDDs create wide dependencies that require communication, but a join can become narrow if both RDDs are hash-partitioned the same way.

Fault tolerance through lineage (12:20)

Spark avoids replicating intermediate results by keeping a lineage: a log of the transformations needed to reconstruct any RDD from the replicated source data on the distributed file system. If a node crashes mid-computation, Spark does not need to have stored the lost partitions - it replays the relevant part of the lineage log to recompute them. The lecture contrasts this with the earlier MapReduce approach of always writing intermediate data to disk, and shows benchmark numbers where Spark's in-memory reuse gives roughly an order-of-magnitude speedup over Hadoop on iterative workloads like logistic regression and k-means.

When scale-out is not worth it (23:32)

The lecture distinguishes scale up (sharing memory across cores in one machine) from scale out (separate machines communicating over a network), and cautions that distributed systems only make sense once a dataset exceeds what a single server's memory can hold - commonly half a terabyte to a few terabytes today. It cites a benchmark showing Spark on 128 cores running roughly twice as slow as a single thread on a dataset that fits in memory, and references a critique that big-data research has overemphasized scalability at the expense of raw performance.

Cache fundamentals recap (27:37)

Shifting to cache coherence, the lecture recaps why caches matter (an off-chip access can cost hundreds of cycles) and reviews the three C's of misses: cold misses on first access, capacity misses when a cache is too small, and conflict misses caused by limited set associativity. Using the Intel Skylake cache hierarchy (private L1 and L2 per core, shared L3) as an example, it explains that higher set associativity lowers the conflict miss rate but makes lookups more expensive, which is why associativity is tuned rather than maximized.

Write policies and cache line metadata (39:51)

The lecture works through cache line structure - data plus metadata including a tag (the line's address) and a dirty bit - and contrasts write-back caches, which only flush a modified line to memory on eviction, with write-through caches, which update memory on every write. It also covers write-allocate, where a write miss first fetches the full line from memory before updating the written word, walking through a concrete write-allocate, write-back example.

The coherence problem (51:02)

Using a trace of several processors reading and writing the same variable through their own private caches, the lecture shows how their cached copies can end up disagreeing with each other and with main memory, even though each individual cache access looks correct. Locks alone cannot fix this, because the problem is that multiple physical copies of the same address exist and are updated independently, not that accesses are unsynchronized. The lecture states the two governing invariants a coherent system must uphold: single-writer/multiple-reader access to any cache line, and a data-value invariant guaranteeing that every reader sees the most recently written value.

Snooping and the MSI protocol (1:06:16)

The lecture rules out a single shared cache (bandwidth bottleneck) and a naive write-through invalidation scheme (every write must broadcast, exhausting bandwidth) before introducing snooping over a bus. A bus gives two properties a coherence protocol can exploit: serialization, since only one transaction happens at a time, and broadcast, since every cache observes every transaction. It introduces the MSI protocol's three cache-line states - modified, shared, invalid - and the processor and bus operations (processor read/write, bus read, bus read-exclusive, bus write-back) that will be used to enforce exclusive write access, with the full protocol walkthrough deferred to the next lecture.

Before you watch

• Review the earlier Spark lectures on RDDs, transformations, and actions, since this lecture assumes that foundation.
• Be comfortable with cache basics from earlier in the course: cache lines, hit/miss, and the three C's of misses.
• Recall the arithmetic-intensity and fusion/tiling locality optimizations from the DNN lecture, since Spark's transformation fusion is presented as the same idea applied to a distributed runtime.

Check your understanding

1. Under what condition can Spark fuse a chain of RDD transformations without any node-to-node communication, and why does a group by key typically break that condition?
2. Why does lineage let Spark recover from a node failure without storing all intermediate data?
3. Give an example of when using a distributed cluster would be slower than just running on a single machine, and explain why.
4. Explain, using the multi-cache trace from the lecture, why locking alone cannot fix the cache coherence problem.
5. State the single-writer/multiple-reader invariant and the data-value invariant in your own words, and explain what a bus's serialization and broadcast properties contribute toward enforcing them.