Lecture 16: Cache Consistency: Memcached at Facebook

What this lecture covers

The lecture walks through the Facebook memcached paper as a case study in scaling a real website: how a single web server evolves into front ends, sharded databases, and finally a caching layer, and what breaks once that caching layer becomes the thing keeping the whole system alive. It treats the paper less as a source of new ideas and more as a record of what a company actually hit when performance requirements collided with consistency.

By the end you should understand why Facebook chose an invalidate-based (lookaside) cache over an update-based one, how leases solve both the thundering herd problem and a subtler stale-data-forever race, and how replication and partitioning trade off across regions, clusters, and a regional pool. The lecture builds on the course's general treatment of caching and replication and applies it to a production system.

Key ideas

• Lookaside cache: the front end, not memcached, is responsible for checking the cache, fetching from the database on a miss, and populating the cache; memcached has no knowledge of the database.
• Invalidate over update: writers send a delete for a key to memcached rather than pushing the new value, because concurrent writers can otherwise leave a stale value permanently cached.
• Leases: memcached hands out a unique token on a miss; only the holder's later set is accepted, which serializes concurrent fills and prevents an old read from clobbering a newer delete.
• Thundering herd: when a popular key is deleted, many front ends miss at once and hammer the database with identical requests; leases fix this by letting only the first misser fetch and install the value while others wait.
• Partition vs. replication: partitioning splits data across servers for more total capacity but scales badly for a few very hot keys; replication serves hot keys in parallel but multiplies memory use.
• Regions, clusters, and the regional pool: Facebook replicates entire regions for locality and failover, shards and lightly replicates within a region via clusters to bound per-request fan-out, and keeps a shared regional pool for unpopular keys to avoid wasting RAM on many copies.
• Cold start and gutter servers: a newly added cluster borrows data from a warm cluster's memcached before touching the database, and gutter servers stand in temporarily when a memcached server fails, both to shield the database from sudden full load.

Walkthrough

From one server to sharded MySQL (2:03)

The lecture traces the typical evolution of a growing website: a single machine running Apache, PHP, and MySQL; then multiple front-end servers sharing one database once PHP CPU time becomes the bottleneck; then sharded MySQL servers once a single database server runs out of capacity. Sharding helps throughput but forces the PHP code to know the shard layout and complicates transactions that span shards.

Web Architecture 3: adding memcached (6:48 approx / chapter "Web Architecture 3")

Even sharded MySQL is expensive per read, and popular keys can overload a single shard no matter how finely data is partitioned. The lecture explains why Facebook inserted a caching layer of memcached servers between front ends and databases: memcached serves reads roughly ten times faster than MySQL for the same hardware, so it absorbs almost all read traffic and lets the database handle only writes and misses.

How reads and writes work with lookaside caching (24:23)

A read asks memcached for the key; on a miss the front end queries the database and installs the result in memcached itself, since memcached knows nothing about the database. A write updates the database, then deletes the key from memcached rather than pushing the new value in. The lecture works through why an update-based scheme (a direct set from the writer) can leave memcached permanently wrong when two clients race to increment the same value, which is why Facebook chose invalidation.

Partition and replication across regions and clusters (38:35)

Facebook fully replicates data across two regions for locality and to keep the hundreds of per-page lookups a front end makes close to home; within a region, data is sharded (not replicated) at the database level, but memcached is both sharded and lightly replicated across clusters so hot keys get parallel service without every cluster growing too large (which would cause costly N-squared connections and incast congestion). A separate regional pool caches unpopular keys just once, since replicating rarely used data everywhere wastes RAM.

Cold start and gutter servers (53:14 and 1:02:26)

Spinning up a brand-new cluster with an empty cache would multiply database load by orders of magnitude, so a cold cluster first checks a warm cluster's memcached before falling back to the database. Separately, when a memcached server fails outright, front ends redirect to idle gutter servers that cache data temporarily (with short timeouts instead of explicit deletes) until the failed server is replaced, so the database is never exposed to a memcached server's full request rate.

Solving the stale-forever race with leases (1:06:34)

The lecture builds a concrete race: a slow reader can install an old value into memcached after a newer write's delete has already passed through, leaving the cache stuck on stale data indefinitely. The fix extends the lease mechanism: a miss grants a lease token, and any delete that arrives afterward invalidates that lease, so a late set carrying an outdated value is rejected by memcached instead of being installed.

Before you watch

• Be comfortable with the idea of a cache and cache invalidation, since the lecture assumes you know what caching buys you before explaining Facebook's specific consistency fixes.
• Some familiarity with earlier lectures on replication and sharding in this course helps, since the lecture treats partition-versus-replication trade-offs as background.

Check your understanding

1. Why does Facebook's write path delete a key from memcached instead of sending the new value directly, and what race does this avoid?
2. How do leases solve the thundering herd problem, and separately, how does the same mechanism prevent a stale value from being cached forever?
3. What trade-off does Facebook make by keeping clusters within a region relatively small rather than one large shared pool of memcached servers?
4. Why does the system use full region-to-region replication instead of sharding data across the two data centers?
5. What role do gutter servers play, and why don't they receive the same delete traffic as ordinary memcached servers?