Lecture 4: Privilege Separation

What this lecture covers

Earlier lectures showed that C programs almost inevitably contain vulnerabilities such as buffer overflows. This lecture asks a different question: instead of trying to eliminate every bug, how do you design a system so that a single bug does not compromise everything? The answer is privilege separation, splitting an application into pieces that each hold only the privileges needed for their job, so a bug in one piece limits the damage an attacker can do.

To make this concrete, the lecture walks through Unix's protection mechanisms (users, groups, file permissions, file descriptors, chroot, setuid) in detail, then applies them to OKWS, a privilege-separated web server built for OkCupid. After watching, you should be able to explain how Unix decides whether a process can act on a file, process, or network resource, and how OKWS's architecture uses those primitives to limit what an attacker can reach after compromising any one component.

Key ideas

• Privilege separation: splitting an application into components that each get only the privileges necessary for their task, so a vulnerability in one component doesn't expose everything.
• Principal: in Unix, a process's privileges are represented by a userid (uid) plus a list of group IDs (gids).
• File permissions: each inode stores an owning uid and gid plus read/write/execute bits for owner, group, and other, often written as an octal number like 644.
• File descriptors: security checks happen when a file is opened; anyone holding the resulting descriptor can use it freely, which lets a privileged helper open a file and hand the descriptor to a less-privileged process.
• setuid binaries: an executable that switches the running process's uid to the file's owner on execution, the classic (and tricky) way to gain elevated privileges in Unix.
• chroot: restricts a process's view of the filesystem to a subtree; only root can call it, and its exact semantics (rewriting how / and /../ resolve) matter for avoiding escapes.
• OKWS architecture: separate processes (okd, pubd, oklogd, services, a database proxy, and the root-only okld launcher) each run under distinct uids and chroots so that compromising one limits an attacker's reach.

Walkthrough

Why privilege separation, and Unix as an isolation mechanism (0:00)

The lecture opens by motivating privilege separation as a general defense: rather than relying on an application having no bugs, split it into pieces so an attacker who exploits one piece only gets that piece's privileges. Real systems use this idea widely: SSH separates components to protect its keys, and Chrome sandboxes its renderer so a bug there doesn't give full control of the machine. Unix is introduced as one mechanism for enforcing this isolation, alongside virtual machines and containers, and the class discusses what needs protecting in an OS: files, directories, sockets, processes, memory, and file descriptors.

Unix principals and file permissions (6:06)

The lecture explains that Unix's security subject is the process, identified by a uid and a set of gids. For files and directories, permissions live in the inode as an owning uid/gid plus a 3x3 matrix of read/write/execute bits for owner, group, and other, commonly encoded as octal (for example 644). Only the file's owner can change its permissions. Directory operations like unlink, link, and rename depend on write permission on the directory rather than the file itself, and execute permission on a directory means the ability to look up names in it. A worked example traces the checks the kernel performs on open("/etc/passwd"), and a puzzle about combining two Unix groups to approximate an intersection shows both the flexibility and the limits of this permission model, including the fact that Unix does not enforce transitive restrictions once a process holds a privilege.

File descriptors, processes, and networking (23:34)

Because file access checks happen only at open time, a file descriptor itself becomes a bearer credential: anyone holding it can read or write freely, and descriptors can be inherited by children or passed over sockets. This lets a privileged process open a file and hand a narrowly scoped descriptor to a less-privileged one. For processes, a process can generally only create, kill, or ptrace another process with the same uid. Networking follows different rules: anyone can connect out, but binding to a port below 1024 requires uid 0 (root), a restriction meant to stop ordinary users from hijacking well-known services like port 80.

Bootstrapping privilege: setuid, login, and chroot (31:46)

The lecture covers how uid values are assigned in the first place: setuid() (callable only by root), the login process that checks credentials against /etc/passwd and /etc/shadow and then calls setuid to drop into the user's account, and setuid binaries such as su, which run with the privileges of the file's owner rather than the invoking user. This mechanism is powerful but historically fragile, for example old bugs let attackers use environment variables to hijack a setuid binary's shared libraries. The lecture then explains chroot, including the two-step mechanism the kernel actually uses (redefining what / means and rewriting /../ within that process), why only root can call it, and a demonstrated escape technique using an open file descriptor to a pre-chroot root directory combined with fchdir and repeated chroot calls.

OKWS: architecture and request flow (50:07)

Contrasted with a typical Apache-style server, where one process handles everything and a single database account has access to all data, OKWS splits a request's handling across several processes: okd (the dispatcher), oklogd (logging), pubd (template rendering), individual services (one per feature, such as newsletters or matching), and a database proxy that only accepts a fixed set of parameterized query templates guarded by a 20-byte token. okld, the launcher, runs as root and is responsible for the operations that require root, such as setting uids, chrooting processes, and binding port 80. The lecture traces how a request flows from the browser through okd to a service and finally to the database proxy.

Evaluating OKWS's security (1:07:40)

The lecture closes by assessing each component's attack surface and blast radius. okd turns out to be the most dangerous component to compromise, since it can rewrite responses and steal other users' passwords and data as they pass through. oklogd mainly risks garbling the log rather than leaking the database. Services are considered the most likely place for bugs, since they're written quickly by individual developers, but their damage is limited to whatever queries the database proxy permits. okld runs as root but accepts very little external input, so its actual attack surface is small. The class also discusses why OKWS separates by service rather than by end user (mainly performance and the fact that services like matching need to read many users' data at once), and confirms that this architecture, while imperfect, is a substantial improvement over an unseparated design.

Before you watch

• Review buffer overflows and C memory-safety vulnerabilities from the prior lecture, since this lecture treats them as the kind of bug privilege separation is meant to contain.
• Be comfortable with basic Unix concepts: users, processes, and the file system.
• If your course lab is modeled on a privilege-separated web server, having read that assignment's description will make the OKWS comparison more concrete.

Check your understanding

1. Why does giving an application fewer privileges per component limit the damage of a single vulnerability, even if that vulnerability is never fixed?
2. Walk through the checks the Unix kernel performs when a process calls open("/etc/passwd").
3. Why can a process holding an open file descriptor read or write it freely, even if it could not have opened that file itself?
4. Explain the trick used to escape a chroot jail using an open file descriptor and fchdir, and why only root is allowed to call chroot.
5. Of okd, oklogd, a service, and okld, which component's compromise is most damaging in OKWS, and why?