Lecture 11: Cache Consistency: Frangipani

What this lecture covers

The lecture studies Frangipani, a distributed file system built for small trusted workgroups, to extract three general ideas: cache coherence protocols, distributed transactions built from locks, and distributed crash recovery with per-workstation write-ahead logs. It's an older paper than the cloud systems the course has been covering, but it reuses machinery (locks, logging, version numbers) that reappears throughout the course's later material on distributed transactions.

After watching, you should be able to explain how Frangipani's lock server drives both cache coherence and atomicity, why it write-ahead logs metadata updates into shared storage rather than local disk, and how version numbers let another workstation safely recover a crashed workstation's log without needing to know who held which locks.

Key ideas

• Split architecture: Frangipani workstations hold all the file-system logic and a write-back cache; the shared storage layer, Petal, is just a replicated virtual disk that knows nothing about files or directories.
• Cache coherence via locks: a workstation may never cache data without holding the corresponding lock, must read from Petal only after acquiring the lock, and must write dirty data back to Petal before releasing it.
• Lock states and revocation: workstations keep locks idle (not actively used but not given back) for performance; a lock server sends a revoke message when another workstation wants a held lock, forcing a write-back before release.
• Atomicity from the same locks: holding every lock needed for a multi-step operation (like a file rename) until all writes are done makes the operation appear instantaneous to everyone else, reusing the coherence machinery for the opposite purpose.
• Per-workstation logs stored in shared storage: each workstation write-ahead logs its metadata changes into its own circular log area in Petal (not on local disk), so another workstation can read and replay it after a crash.
• Version numbers make replay idempotent and safe: every piece of metadata and every log entry carries a version number, so recovery only replays a log entry if the stored data's version is not already at least as new—this lets recovery proceed without acquiring any locks.
• Periodic flush for durability: modified cache contents are written back at least every 30 seconds regardless of locking activity, bounding data loss on a crash.

Walkthrough

System overview: workstations, Frangipani, and Petal (0:00)

Frangipani is designed to look like an ordinary shared file system for a group of trusted users on personal workstations, similar in spirit to AFS. Each workstation runs a Frangipani module that intercepts file system calls and implements caching, including write-back caching: creating a file, for instance, is initially done entirely in local memory and cache, and is not sent to the shared Petal storage service until necessary. Petal itself is treated as a simple replicated virtual disk, addressed by block number, with no knowledge of files.

Three challenges from caching (10:18)

Write-back caching creates three problems the rest of the lecture addresses: cache coherence (a file created on one workstation must become visible to others, even though it was only written locally at first), atomicity (concurrent operations on the same directory from different workstations must not corrupt it), and crash recovery (a workstation crashing mid-operation must not leave the shared file system in a broken state for everyone else).

Cache coherence via locks (18:22)

A separate lock server maintains a table of named locks, one per file or directory (named by inode number). The coherence rule is strict: a workstation may only cache data it holds the lock for, must acquire the lock before reading from Petal, and must write dirty data back to Petal before releasing the lock. Workstations keep locks in an "idle" state after finishing an operation, rather than releasing immediately, since they're likely to reuse the same files soon. When another workstation requests a held lock, the lock server sends a revoke message; the holder writes back any dirty data and then releases. A worked example shows two workstations exchanging a lock for a file: the reader can't see the data until the writer has flushed its modifications to Petal.

Atomicity from held locks (39:56)

Multi-step operations like renaming a file are made atomic by acquiring every lock the operation needs before making any changes, and not releasing any of them until all the writes are complete. Because the coherence protocol already prevents other workstations from seeing cached data whose lock hasn't been released, this automatically hides partial updates—an elegant reuse of the same lock infrastructure for what is effectively the opposite purpose (hiding writes during an operation, rather than exposing them promptly afterward).

Write-ahead logging for crash recovery (45:59)

Before writing any modified block back to Petal, a workstation must first append a complete log entry describing the operation to its own log, which is stored in Petal itself (not local disk) precisely so another workstation can read it after a crash. Each workstation has its own log with increasing sequence numbers; log entries record only metadata changes (inode and directory updates), not file contents. When a workstation gets a revoke for a lock, it flushes its log, then writes back only the blocks covered by that lock, then releases.

Recovering a crashed workstation's log (59:19)

If a workstation stops responding to a revoke, its lock lease eventually expires and the lock server asks a live workstation to replay its log before releasing its locks. Because writes are always logged before being applied, a crash can only occur before any log write, mid-log, mid-page-write (after the whole log entry is durable), or after everything completes—so recovery can always safely replay whatever complete log entries exist.

The version-number problem and its fix (1:06:29)

A subtler case: if a crashed workstation deleted a file and, after it released the lock, a different workstation created a new file with the same name, blindly replaying the crashed workstation's stale log entry could wrongly delete the new file. Frangipani solves this by tagging every piece of metadata and every log entry with a version number; recovery only replays a log entry if the stored data's current version is not already at least as high. This also lets recovery proceed without acquiring locks at all, since a released lock implies the data was already written back with a high enough version number, making replay safe even after a total power failure that erases all knowledge of who held what.

Before you watch

• Review how write-ahead logging and undo/redo recovery work, covered in the Aurora lecture, since Frangipani uses the same core idea for its own logs.
• Be comfortable with the general notion of a lock server and distributed locking from the Zookeeper lectures.
• No specific lab dependency, but familiarity with how replicated storage like Petal (a chain-replication-style virtual disk) fits underneath a client is helpful context.

Check your understanding

1. Why must a workstation write dirty cached data back to Petal before it can release the corresponding lock?
2. How does holding all of an operation's locks until every write is finished make multi-step operations like rename appear atomic to other workstations?
3. Why does Frangipani store each workstation's write-ahead log in shared Petal storage rather than on the workstation's local disk?
4. Walk through why replaying a crashed workstation's log entries in order isn't always safe, and how version numbers fix the problem.
5. Why can recovery software read and write Petal data without acquiring any locks, even after a crash that erases lock state?