Lecture 12: Memory Consistency

What this lecture covers

The lecture completes the course's treatment of cache coherence - walking through the MSI protocol's state transitions in detail, adding the MESI exclusive state, and introducing directory-based coherence as a scalable alternative to bus snooping - along with the performance consequences of coherence traffic, including false sharing. It then turns to memory consistency: the separate question of how reads and writes to different addresses, made by different processors, appear ordered to each other.

After watching, you should be able to trace how a cache line moves between invalid, shared, and modified states in response to processor and bus actions, explain why directories scale better than snooping over a bus, recognize false sharing and why it hurts performance, state what sequential consistency guarantees, and explain why relaxed consistency models (enabled by mechanisms like write buffers) require programmers to use synchronization and fences to get predictable behavior.

Key ideas

• MSI state transitions: a processor read on an invalid line triggers a bus read and moves it to shared; a processor write triggers a bus-read-exclusive and moves it to modified; seeing another processor's bus-read-exclusive on a line you hold invalidates it (from shared) or forces a bus write-back and invalidation (from modified).
• MESI's exclusive state: adding a "clean, exclusive" state lets a processor upgrade a line it exclusively holds to modified without an extra bus transaction, avoiding the double-miss cost of a plain MSI upgrade.
• Directory-based coherence: instead of broadcasting every transaction to every cache over a bus, a directory (often tracked alongside a shared last-level cache) records which caches hold each line, so invalidations go only to processors that actually need them - this scales to many more cores than snooping.
• NUMA and coherence latency: accessing a line that is shared, or modified in another core's cache, costs noticeably more cycles than a private hit, and multi-socket (NUMA) systems add further latency for accessing another socket's local memory.
• False sharing: independent variables that happen to land on the same cache line generate coherence traffic as if they were shared, which can slow a program down by several times; padding or restructuring data can fix it.
• Coherence vs. consistency: coherence only governs the order of accesses to a single address; consistency governs the apparent order of accesses to different addresses across processors, and matters even in systems without caches.
• Sequential consistency: Leslie Lamport's model requires that all operations appear to execute in some single global sequential order, and that each processor's own operations appear in that order in program order - it forbids the reordering that write buffers rely on for performance.
• Relaxed models and fences: write buffers, and models like total/partial store order, let writes be delayed or reordered relative to other accesses for performance, which is why data-race-free programs must use synchronization (which is built on hardware fences) rather than relying on raw shared-memory ordering.

Walkthrough

MSI protocol transitions in detail (0:05)

The lecture reviews the two coherence invariants (single-writer/multiple-reader, and data-value) and then works through the MSI state-transition diagram action by action: a processor read on an invalid line issues a bus read and moves to shared; a processor write on an invalid or shared line issues a bus-read-exclusive and moves to modified; hits in modified state need no bus transaction. It also covers what a cache must do when it snoops another processor's transaction on the bus - for example, moving from modified to invalid (with a bus write-back to supply the data) when another processor issues a bus-read-exclusive - and works through a multi-cache trace to reinforce the states, bus transactions, and where each read's data comes from.

MESI: adding an exclusive state (26:31)

Plain MSI forces an "upgrade" from shared to modified to cost two bus transactions (a miss to read, then a miss to gain exclusive access), even when no other cache actually holds the line. MESI fixes this by adding an exclusive state: reading a line that no other cache holds brings it in as clean-but-exclusive rather than shared, so a later write can upgrade it to modified as a cache hit with no bus transaction.

Directory-based coherence (30:36)

Bus snooping requires every transaction to be broadcast to and checked by every cache, which limits scalability. The lecture introduces directories, which record which caches hold each cache line (for example, as presence bits plus a dirty bit stored alongside a shared, inclusive last-level cache) so that invalidations and data transfers only go to the processors that need them. This lets coherence run over non-broadcast interconnects like rings, and is the approach used in most modern multicore chips.

Performance impact: NUMA and false sharing (34:39)

Coherence traffic changes the distribution of memory access latencies: a shared or another-core-modified cache line costs noticeably more cycles to access than a private hit, and NUMA systems add extra latency when a core accesses memory local to a different socket. The lecture then demonstrates false sharing, where independent per-thread counters placed adjacently in memory land on the same cache line and trigger coherence traffic as if they were truly shared - an example showed a roughly threefold slowdown from this alone. It also reviews benchmark data showing that while larger cache lines reduce true-sharing misses through better spatial locality, they can increase false-sharing misses, so the two effects trade off against each other.

From coherence to consistency (53:05)

The lecture pivots to memory consistency, which governs the order in which reads and writes to different addresses, made by different processors, are allowed to appear. Unlike coherence, consistency matters even in a system with no caches at all, because it defines what a shared-memory program is allowed to mean. The lecture works through a two-processor example (each sets one variable and reads the other) and uses a happens-before argument to show that certain outputs are impossible under any reasonable memory model, motivating the formal notion of sequential consistency: all operations appear to execute in a single global order, consistent with each processor's own program order.

Relaxing consistency for performance (1:07:26)

Sequential consistency is intuitive but forbids the write buffers that real processors use to hide write latency, because a buffered write lets a later read to a different address complete first. The lecture shows how this can produce outputs sequential consistency would forbid, and introduces weaker models - total store order and partial store order - that formalize which reorderings (write-to-read, and eventually write-to-write) are allowed in exchange for better performance. ARM's relaxed model permits reordering all four combinations, making explicit memory fences necessary wherever an ordering guarantee is actually required.

Data-race-free programming (1:16:33)

The lecture closes on the practical implication: an unsynchronized program with two accesses to the same address, at least one a write, has a data race and can behave unpredictably under a relaxed consistency model. The fix is data-race-free programming - always synchronizing shared-data accesses - so that application programmers can rely on synchronization libraries (whose authors must understand the underlying hardware's consistency model and use fences correctly) rather than reasoning about raw memory ordering themselves.

Before you watch

• Watch the previous lecture in this course, which introduces the cache coherence problem and the two governing invariants that this lecture builds on.
• Be comfortable with basic cache terminology (cache lines, hits/misses, dirty bits) from earlier in the course.
• Recall bus properties (serialization, broadcast) since directories are presented as a way to avoid relying on them.

Check your understanding

1. Walk through what happens, state by state and bus transaction by bus transaction, when one processor reads a line, then a second processor writes to it, then the first processor reads it again.
2. Why does MESI's exclusive state remove a bus transaction that plain MSI would require, and in what situation does that saving apply?
3. Why does a directory scale better than bus snooping, and what information does it need to track?
4. Give an example of false sharing and explain why padding a data structure can fix it.
5. Explain why a write buffer breaks sequential consistency, and why a programmer writing correctly synchronized code usually does not need to think about this directly.