Lecture 7: Sandboxing Native Code

What this lecture covers

This lecture introduces Native Client (NaCl), the system Google built into Chrome to let web applications run native x86 code alongside JavaScript. It starts by asking why anyone would want native code in a browser at all (performance, reusing existing C/C++ libraries, and freedom from JavaScript), then contrasts NaCl with earlier, weaker approaches like trusting a signed binary (ActiveX) or relying on OS-level sandboxes such as Capsicum or seccomp. NaCl instead uses software fault isolation: it disassembles a binary ahead of time, proves that every instruction it could ever execute is safe, and only then allows it to run, without trusting the operating system's own isolation mechanisms.

After watching, you should be able to explain why variable-length x86 instructions make binary verification hard, how NaCl's alignment and instrumentation rules solve that problem, and how x86 segmentation hardware lets NaCl confine an untrusted module's memory accesses cheaply. This lecture builds on the two previous lectures' treatment of Unix privilege separation (OKWS) and Capsicum, and applies the same underlying question, how to limit what a compromised or malicious component can do, to a very different mechanism.

Key ideas

• Software fault isolation (SFI): a sandboxing technique that checks a binary's instructions ahead of time rather than relying on the OS or hardware to catch bad behavior at runtime.
• Safe vs. unsafe instructions: arithmetic and register operations need no checks; memory accesses and jumps are unsafe and must be instrumented; system calls and other privileged instructions are simply disallowed.
• Reliable disassembly: because x86 instructions have variable length, a naive left-to-right scan can be fooled by jumping into the middle of an instruction; NaCl solves this by requiring every jump target to be an address actually seen during that scan.
• 32-byte alignment and pseudo-instructions: indirect jumps are prefixed with an and that masks the target to a multiple of 32 bytes, and every 32-byte-aligned address must start a valid instruction, so an attacker cannot jump into the middle of one.
• Trusted service runtime: the only code allowed to perform genuinely unsafe operations (memory allocation, threads, communicating with the browser); untrusted modules reach it only through fixed entry points called trampolines.
• x86 segmentation: a legacy hardware mechanism (base and length per segment) that NaCl repurposes to confine an untrusted module's code and data references to a 256MB region, cheaply and without extra instructions on every access.
• Trampolines and springboards: fixed, 32-byte-aligned code paths (supplied by the trusted runtime, never the module) used to leave the sandbox safely and to re-enter it with the memory bounds correctly restored.

Walkthrough

Introduction to Native Client (0:00)

The lecture opens by framing NaCl as a real system used in Chrome, and as an example of software fault isolation, a sandboxing approach that doesn't rely on the operating system or virtual machines.

Motivation for native code (1:09)

The class discusses why a browser would want to run raw native code at all: performance (native code can run as fast as or faster than JavaScript), reusing existing "legacy" libraries such as performance-sensitive graphics engines, and freedom to use languages other than JavaScript.

Native Client demo (3:14)

A live demo shows a minimal web page with a JavaScript wrapper that posts a message to an embedded NaCl module, which replies with a string that pops up in an alert box. A deliberately introduced buffer overflow in the module's C++ code crashes the module (reported in the browser's console) without crashing the rest of the browser, illustrating that NaCl contains memory corruption within the sandbox.

Alternative sandboxing approaches (7:39)

Before diving into NaCl's mechanism, the lecture surveys weaker alternatives: trusting the developer or asking the user for permission (as with signed ActiveX controls, which say nothing about whether the code is actually safe), and OS- or hardware-level sandboxing such as Capsicum or Linux's seccomp. NaCl avoids OS-level sandboxes because it doesn't want to trust a large, unauditable kernel, because different operating systems expose incompatible isolation mechanisms (forcing different code for each platform), and for performance, since it doesn't need to fork a separate process per OS-level primitive. The lecture also notes that NaCl's designers found real hardware bugs where malformed instructions could hang a CPU instead of trapping safely into the kernel, motivating a design that doesn't even fully trust the processor's own protections.

Software fault isolation design (17:23)

NaCl's core idea is explained: scan every instruction in a binary ahead of time and classify it as safe (arithmetic, register moves), unsafe-but-necessary (memory access, which gets instrumented with a bounds check), or simply disallowed (system calls, privileged instructions). Once every instruction has passed this check, the module can run because, by construction, nothing unsafe can execute. Because a fully sandboxed module can't touch the network, disk, or display, NaCl pairs it with a trusted service runtime, written by Google, that implements the small set of privileged operations (memory allocation, threads, message passing to the browser) the module is allowed to invoke through fixed entry points.

Defining safety and memory bounds (24:32)

Safety in NaCl means two things: no disallowed instructions can ever execute, and all code and data accesses stay within a dedicated 256MB region of the process's address space, starting at address zero. The lecture discusses why starting at zero matters mainly for performance, since checking that an address falls in a zero-based, power-of-two-sized region can be done with a single bitwise and.

Reliable disassembly techniques (30:23)

A naive left-to-right scan for disallowed instructions can be defeated because x86 instructions are variable-length: a byte sequence that looks safe when read from one offset can decode as a dangerous instruction (like a system-call trap) if execution actually starts one byte later. NaCl's fix is to guarantee that the CPU can never jump to an offset the verifier didn't already see during its own left-to-right scan, which reduces the problem to controlling every possible jump target.

Control flow verification (44:26)

Direct jumps are checked against the set of addresses seen during disassembly. Indirect jumps (through a register, needed for function pointers) can't be checked that way, so NaCl's compiler emits a pseudo-instruction: an and that masks the target to a multiple of 32 bytes, immediately followed by the jump, treated by the validator as a single atomic unit so an attacker can't jump into the middle of it. The lecture works through why 32 bytes was chosen (a power of two, at least as long as the longest x86 instruction the compiler emits, and small enough to avoid wasting memory on padding), and why ret is disallowed in favor of controlled jumps, since checking a value popped off a writable stack would be vulnerable to a race between threads.

Validator rules and security (51:53)

The lecture walks through the paper's table of validator rules (C1 through C7): the loaded binary is non-writable, it must be linked to start at a fixed layout, indirect jumps must use the guarded pseudo-instruction, the binary is padded to a page boundary with halts, no instruction may span a 32-byte boundary, every instruction must be reachable from the start via disassembly, and direct jump targets must be valid. A discussion of a real historical NaCl bug shows what goes wrong if the validator ever miscalculates an instruction's length: an attacker can hide a system-call instruction inside the bytes the validator skips over but the CPU still executes.

Hardware-assisted memory limits (1:00:57)

Rather than instrumenting every memory access in software (costly, since x86 addressing modes are complex), NaCl uses x86 segmentation hardware: a per-process table of segment descriptors, each with a base and length, that the CPU consults on every memory reference. NaCl points the code and data segment selectors at an entry with base zero and a 256MB length, so the hardware itself confines all memory accesses without extra instructions. Because segment selector registers are otherwise ordinary and writable, NaCl must also disallow instructions that load a new value into them, or an untrusted module could switch back to the OS's unrestricted segment and escape.

Trampolines, springboards, and leaving the sandbox (1:14:33)

To call into the trusted runtime, an untrusted module jumps to a trampoline, fixed, 32-byte-aligned code supplied by the trusted runtime (never by the module itself) that resets the segment selectors to the runtime's unrestricted segment before jumping into privileged code. Returning is handled by a springboard, which restores the restricted segment selectors before jumping back into the module; a halt instruction at the start of the springboard's aligned slot prevents the untrusted module from jumping into it directly and bypassing the restriction.

Conclusion (1:23:03)

The lecture wraps up by noting that this covers the essentials of the NaCl mechanism, with web security as the topic for the next lecture.

Before you watch

• Review the earlier lectures on Unix privilege separation (OKWS) and capabilities (Capsicum), since this lecture repeatedly compares NaCl's approach to OS- and capability-based sandboxing.
• Basic familiarity with x86 assembly (registers, jump instructions, the stack) and how buffer overflows work will make the instruction-level discussion easier to follow.
• A general sense of how a browser loads and runs JavaScript is useful context for the demo.

Check your understanding

1. Why can't a naive left-to-right scan of a binary's bytes reliably catch every disallowed instruction on x86?
2. Explain the role of the and-then-jump pseudo-instruction and why the validator treats it as a single atomic unit.
3. Why does NaCl disallow the ret instruction, and what race condition would checking a stack-based return address be vulnerable to?
4. How does x86 segmentation hardware let NaCl confine memory accesses to a 256MB region without instrumenting every load and store?
5. Why must trampolines and springboards be supplied by the trusted runtime rather than by the untrusted module itself?