Lecture 9: ISA and Microarchitecture

What this lecture covers

This lecture closes out the course's discussion of instruction set architecture (ISA) and opens the transition into microarchitecture. It answers a specific question: which properties of a processor must be written into the ISA specification, and which are free choices left to the hardware designer? Along the way it introduces indirection as a general design principle and contrasts the familiar control-flow (von Neumann) execution model with an alternative data-flow model.

The lecture sits between the course's earlier material on instruction formats, addressing modes and the LC-3b state machine, and the multi-cycle and pipelined microarchitectures built in later lectures. After watching, you should be able to classify a processor feature as ISA-level or microarchitecture-level, explain why indirection lets designers change performance trade-offs without changing software, and read a simple data-flow graph.

Key ideas

• Semantic gap: the distance between an ISA's instructions and a high-level language; a small gap gives hardware designers more room to optimize, a large gap (instructions close to raw gates) pushes optimization work onto software.
• Indirection: David Wheeler's principle that adding a translation layer changes a system's trade-offs at the cost of extra complexity and latency; used for x86-to-internal-micro-op translation, Apple's Rosetta-style translation, NVIDIA Denver, and historically Transmeta.
• Secret micro-operations: the internal, simpler instruction set that x86 processors translate complex instructions into before execution, so the visible ISA can stay complex while the hardware stays simpler.
• Register count trade-off: more architectural registers reduce spills and improve compiler register allocation, but enlarge instruction encodings and increase register file access time, area and power.
• Control-flow execution: instructions are fetched and executed in program-counter order; this is the dominant model and the one the course otherwise assumes.
• Data-flow execution: an instruction (data-flow node) fires as soon as all of its input operands are available, with no program counter; many nodes can fire concurrently.
• ISA vs. microarchitecture: the ISA is the agreed hardware/software interface (what must be documented and obeyed); microarchitecture is a specific implementation of that interface, invisible to software as long as it preserves ISA semantics.
• Design point: architecture is described as making trade-offs among cost, energy, performance and time-to-market for a given application and market space, since future workloads cannot be fully predicted.

Walkthrough

Picking up ISA properties and the instruction processing cycle (8:48)

The lecture opens by recapping instruction types (operate, data movement, control), the one-Neumann model's stored-program and sequential-processing properties, and the six-step instruction processing cycle used to describe LC-3b. Mutlu stresses that the state-machine implementation shown earlier for LC-3b is only one possible microarchitecture, and that later lectures will build better ones with different principles.

The semantic gap between complex and simple instructions (12:51)

The lecture frames a core ISA design choice: instructions can be complex (doing a lot of work per instruction, closer to high-level constructs like matrix multiply) or simple (close to hardware primitives like AND, OR, NOR). A small semantic gap between the ISA and the high-level language shifts optimization burden onto hardware designers, who gain more room to implement an operation in different ways; a large gap shifts the burden onto compilers, assembly programmers, or language models.

Indirection: translating one ISA into another (15:54)

Using David Wheeler's principle that any problem can be solved with another level of indirection, the lecture explains how processors keep a complex, backward-compatible visible ISA (like x86-64) while translating it in hardware and software to a simpler internal instruction set of secret micro-operations. Examples given include Apple Silicon's x86 translation, NVIDIA Denver's ARM-to-internal translation, IBM's early 1970s work, and the historical Transmeta processor, which translated x86 to an internal VLIW ISA via "code morphing" software. Indirection is described as adding new optimization opportunities but also extra latency and complexity; virtual memory is named as a later example of the same principle.

Register count and other ISA-level trade-offs (26:06)

The lecture walks through how many general-purpose registers an ISA should expose, contrasting LC-3's eight registers (3-bit encoding) with MIPS's 32 (5-bit encoding). More registers improve register allocation and reduce memory traffic but enlarge instructions and register file cost. It notes that indirection can also apply here: an ISA can expose few architectural registers while the microarchitecture internally maps them onto a larger physical register file, a technique used later in out-of-order execution.

Data-flow execution versus control-flow execution (30:08)

The lecture deconstructs the one-Neumann assumption of sequential, program-counter-driven execution by introducing the data-flow model, where an instruction fires only when all its operand values are available, with no program counter or instruction pointer. It shows graphical data-flow nodes for arithmetic, conditional branching, relational comparison, and barrier synchronization, then walks through a factorial computation expressed as a data-flow graph, tracing how tokens propagate and multiple nodes fire concurrently. The section closes by noting that although data flow was historically hard to program directly, modern out-of-order processors internally convert sequential programs into something resembling a data-flow graph without exposing this to software.

Defining ISA and microarchitecture, and testing the boundary (1:09:00)

The lecture gives Gene Amdahl's classic definition of architecture as what the programmer sees, distinct from microarchitecture as the organization of data flow, control logic and physical implementation. It uses a car analogy (pedals and steering wheel as the stable interface, engine internals as the changeable implementation) and the add instruction (fixed opcode, but many possible adder circuits) to illustrate that few ISAs exist but many microarchitectures implement each one. A quick-fire classification game follows, sorting properties such as opcode, adder type, register count, multiply latency, register file port count, and pipelining into ISA or microarchitecture, with the conclusion that most implementation details are microarchitectural unless explicitly documented.

From single-cycle to multi-cycle instruction processing (1:41:26)

Closing the lecture, Mutlu introduces the basic finite-state-machine view of instruction processing: architectural state transforms into new architectural state according to ISA semantics. He contrasts a single-cycle implementation, where the slowest instruction sets the clock period, with a multi-cycle implementation that updates microarchitectural (not architectural) state across several shorter cycles, only exposing the final architectural state update to software. This sets up the single-cycle and multi-cycle microarchitecture designs built in the following lectures.

Before you watch

• Be comfortable with the one-Neumann model, instruction types, and the LC-3b instruction processing cycle from earlier lectures in this course.
• Review combinational logic and critical-path timing analysis, since the lecture reuses those ideas to explain clock cycle time.
• Familiarity with MIPS or LC-3 register counts and addressing modes helps the register trade-off discussion make sense.

Check your understanding

1. Why does adding a level of indirection, such as translating x86 to internal micro-operations, let a hardware designer simplify the physical implementation without changing the software-visible ISA?
2. In the data-flow model, what determines when an instruction can execute, and how does this differ from control-flow execution?
3. For each of the following, is it typically part of the ISA or the microarchitecture: the opcode of an add instruction, the type of adder circuit used, the number of general-purpose registers, and the number of cycles a multiply instruction takes?
4. Why does a single-cycle processor's clock period get determined by its slowest instruction, and how does a multi-cycle design avoid that constraint?