Seyed Masoud Hosseini · Overview · Study log · Weekly summaries · Ideas · Search · Transcript · RSS feed

Digital Design & Computer Architecture · Lecture 23 of 37 · 11:48

Lecture 18b: Decoupled Access-Execute

Study guide

What this lecture covers

This short, optional lecture introduces decoupled access-execute (DAE), a 1980s alternative to Tomasulo's algorithm for extracting instruction-level parallelism without the complexity of full out-of-order execution. The motivating question is how a machine can overlap memory access and computation, letting the processor keep working on one while the other stalls, without building expensive reservation stations and large physical register files.

It sits alongside the course's coverage of VLIW and systolic arrays as a third historical execution paradigm, and closes the sequence on execution models before the course moves on to SIMD and GPU architectures. After watching, you should be able to explain how access and execute queues replace out-of-order hardware structures, why branches complicate this split, and where the same decoupling principle shows up inside modern out-of-order processors.

Key ideas

Decoupled access-execute (DAE): splits a program into two instruction streams, one issuing memory accesses and one performing computation, that communicate through hardware FIFO queues rather than shared registers.
ISA-visible queues: the queues are part of the instruction set, and their length limits how much memory or execution latency the machine can tolerate before stalling.
Scalable communication: FIFO queues are cheaper to scale than the tag-matching and reservation-station logic used in out-of-order machines.
Branch synchronization: because a single program is split into two streams, branches executed by the execute processor must be signaled back so the access processor does not run down the wrong path.
Loop unrolling: replicating a loop body multiple times per iteration reduces branch and loop-overhead instructions, which helps DAE, VLIW, and systolic-array designs alike.
Modern relevance: contemporary out-of-order processors (the lecture uses Pentium 4 as an example) still separate memory and execution pipelines internally, echoing the DAE idea even though it is not built exactly as originally proposed.

Walkthrough

Basic idea: decoupling access from execute (0:00)

The lecture opens with the motivation: Tomasulo's algorithm was seen as too complex to implement in the 1980s, before Pentium Pro. Jim Smith's 1982 paper proposed instead splitting a program into an access processor, which fetches memory data and supplies it to computation, and an execute processor, which computes and sends back the addresses it needs. The two communicate only through ISA-visible FIFO queues, so the machine gets partial out-of-order behavior between access and execution without reservation stations or large register files.

Advantages, disadvantages, and the ZS1 example (3:23)

The execute stream can run ahead of the access stream and vice versa, so whichever side is not currently bottlenecked keeps doing useful work. The queues scale more cheaply than reservation stations or load-store queues. The compiler is important here, much as it is for VLIW, since it must partition the program and manage the queues; this determines how much decoupling is actually achieved. The Astronautics ZS1 processor is used as a concrete example: it takes a single instruction stream and dynamically splits it into separate in-order access and execute pipelines that communicate through queues and a copy unit.

Branch handling via loop unrolling (6:27)

Splitting one instruction stream into two creates a synchronization problem for branches: they execute on the execute processor but must be signaled back so the access processor does not stray onto the wrong path. Compilers mitigate this with loop unrolling, replicating the loop body several times per iteration to shrink the number of branches and loop-control instructions, enlarge basic blocks for better scheduling, and reduce loop-maintenance overhead. The tradeoff is extra code needed to handle iteration counts that are not a multiple of the unroll factor.

Conclusion: decoupled access-execute in modern processors (9:49)

The lecture argues that DAE's principle, separating memory operations from execution operations into specialized, decoupled components, is present in essentially every modern out-of-order processor, illustrated with the Pentium 4's internal memory and execution portions after renaming and allocation. A student question about combining DAE with VLIW prompts the instructor to note that giving up some strict VLIW lock-stepping to add access/execute queues could recover partial out-of-order benefits without the full hardware cost of true out-of-order execution.

Before you watch

Watch (or recall) the systolic-array and VLIW lectures in this course, since this lecture repeatedly compares DAE against both.
A basic sense of out-of-order execution and Tomasulo's algorithm helps, since DAE is presented as a simpler alternative to it.

Check your understanding

What problem was decoupled access-execute originally designed to solve, and what alternative approach was it competing against?
Why are the access-execute queues described as more scalable than reservation stations?
Why do branches require special synchronization between the access and execute processors, and how does loop unrolling help?
In what sense does a modern out-of-order processor still embody the decoupled access-execute idea?

Vocabulary

decoupled access-execute (DAE) (noun): A design that splits a program into a memory-access stream and a computation stream that communicate through queues.
Decoupled access-execute overlaps memory access with computation.
FIFO queue (noun): A first-in, first-out data structure where items leave in the same order they entered.
The access and execute streams communicate through a FIFO queue.
access processor (noun): The part of a decoupled design responsible for fetching memory data.
The access processor supplies data to the execute processor through a queue.
execute processor (noun): The part of a decoupled design responsible for performing computation.
The execute processor computes results and sends back needed addresses.
scalable (adjective): Able to grow in size or capability without becoming much more expensive or complex.
FIFO queues are more scalable than reservation stations.
synchronization (noun): Coordinating two or more processes so they stay correctly aligned with each other.
Branch synchronization keeps the two streams from diverging incorrectly.
loop unrolling (noun): A compiler technique that repeats a loop's body several times to reduce loop overhead.
Loop unrolling reduces the number of branches executed.
loop overhead (noun): The extra instructions needed to manage a loop, like counting or branching, beyond its useful work.
Unrolling the loop cuts down on loop overhead.
partition (verb): To divide something into separate parts.
The compiler must partition the program into access and execute streams.
in-order pipeline (noun): A processor pipeline that processes instructions strictly in their original order.
The ZS1 splits one instruction stream into two in-order pipelines.
copy unit (noun): Hardware that moves data between the separate access and execute streams.
A copy unit connects the access and execute pipelines in the ZS1.
instruction-level parallelism (noun): The amount of work in a program that independent instructions can do at the same time.
DAE extracts instruction-level parallelism without full out-of-order hardware.
out-of-order execution (noun): Running instructions in a different order than written when it is safe and faster.
DAE was proposed as a simpler alternative to full out-of-order execution.
reservation station (noun): A hardware buffer that holds an instruction until its operands are ready.
DAE avoids the cost of building reservation stations.
register file (noun): A small, fast group of storage locations inside a processor used to hold values during computation.
Full out-of-order machines need large physical register files.
overlap (verb): To happen at the same time as something else, partly or fully.
Access and execute streams overlap memory access with computation.
run ahead (phrasal verb): To proceed further forward than another related process.
The execute stream can run ahead of the access stream.
in flight (idiom): Currently being processed but not yet finished.
Several memory requests can be in flight at once.
basic block (noun): A straight-line sequence of instructions with no branches in or out except at the start and end.
Loop unrolling enlarges basic blocks for better scheduling.
renaming (noun): Assigning a temporary internal name to a value to avoid false conflicts between instructions.
Register renaming happens before instructions reach the execution units.
lock-step (adjective): Moving together in exact coordination, with no independent timing.
Strict VLIW lock-step scheduling leaves no room for stalls.
stray (verb): To move away from the intended or correct path.
The access processor must not stray onto the wrong path after a branch.
mitigate (verb): To make a problem less severe.
Compilers mitigate branch synchronization issues with loop unrolling.
historical (adjective): Belonging to or connected with the past.
DAE is presented as a historical execution paradigm from the 1980s.
bottlenecked (adjective): Slowed down because of one limiting part of a system.
Whichever side is not currently bottlenecked keeps doing useful work.

Chapters

0:00 Basic Idea
3:23 Advantages Disadvantages
6:27 Branch Handling
9:49 Conclusion

From the YouTube description

Digital Design and Computer Architecture, ETH Zürich, Spring 2025 (https://safari.ethz.ch/ddca/spring2025/)

Lecture 18b: Decoupled Access-Execute
Lecturer: Prof. Onur Mutlu
Date: 2 May 2025

Lecture 18b Slides (pptx): https://safari.ethz.ch/ddca/spring2025/lib/exe/fetch.php?media=onur-ddca-2025-lecture18b-dae-beforelecture.pptx
Lecture 18b Slides (pdf): https://safari.ethz.ch/ddca/spring2025/lib/exe/fetch.php?media=onur-ddca-2025-lecture18b-dae-beforelecture.pdf

Recommended Reading:
====================
Intelligent Architectures for Intelligent Computing Systems
https://people.inf.ethz.ch/omutlu/pub/intelligent-architectures-for-intelligent-computingsystems-invited_paper_DATE21.pdf

A Modern Primer on Processing in Memory
https://people.inf.ethz.ch/omutlu/pub/ModernPrimerOnPIM_springer-emerging-computing-bookchapter21.pdf

RowHammer: A Retrospective
https://people.inf.ethz.ch/omutlu/pub/RowHammer-Retrospective_ieee_tcad19.pdf

RECOMMENDED LECTURE VIDEOS & PLAYLISTS:
========================================
Computer Architecture Fall 2021 Lectures Playlist:
https://www.youtube.com/watch?v=4yfkM_5EFgo&list=PL5Q2soXY2Zi-Mnk1PxjEIG32HAGILkTOF

Computer Architecture Fall 2022 Lectures Playlist:
https://www.youtube.com/watch?v=BIpPTqHK-Lc&list=PL5Q2soXY2Zi-cAls3cyauNzM7-74Eq31O

Digital Design and Computer Architecture Spring 2022 Livestream Lectures Playlist:
https://www.youtube.com/watch?v=cpXdE3HwvK0&list=PL5Q2soXY2Zi97Ya5DEUpMpO2bbAoaG7c6

Digital Design and Computer Architecture Spring 2021 Livestream Lectures Playlist:
https://www.youtube.com/watch?v=LbC0EZY8yw4&list=PL5Q2soXY2Zi_uej3aY39YB5pfW4SJ7LlN

Featured Lectures:
https://www.youtube.com/watch?v=jVYCchBGNVc&list=PL5Q2soXY2Zi8VrmOTz44l2WupethSdh-M&index=1

Interview with Professor Onur Mutlu:
https://www.youtube.com/watch?v=8ffSEKZhmvo&list=PL5Q2soXY2Zi8VrmOTz44l2WupethSdh-M&index=9

The Story of RowHammer Lecture:
https://www.youtube.com/watch?v=sgd7PHQQ1AI&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=39

Accelerating Genome Analysis Lecture:
https://www.youtube.com/watch?v=r7sn41lH-4A&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=41

Memory-Centric Computing Systems Tutorial at IEDM 2021:
https://www.youtube.com/watch?v=H3sEaINPBOE&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=35

Intelligent Architectures for Intelligent Machines Lecture:
https://www.youtube.com/watch?v=GTieZPY4Wmc&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=38

Computer Architecture Fall 2020 Lectures Playlist:
https://www.youtube.com/watch?v=c3mPdZA-Fmc&list=PL5Q2soXY2Zi9xidyIgBxUz7xRPS-wisBN

Digital Design and Computer Architecture Spring 2020 Lectures Playlist:
https://www.youtube.com/watch?v=AJBmIaUneB0&list=PL5Q2soXY2Zi_FRrloMa2fUYWPGiZUBQo2

Public Lectures by Onur Mutlu, Playlist:
https://www.youtube.com/watch?v=kgiZlSOcGFM&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl

Computer Architecture at Carnegie Mellon Spring 2015 Lectures Playlist:
https://www.youtube.com/watch?v=zLP_X4wyHbY&list=PL5PHm2jkkXmi5CxxI7b3JCL1TWybTDtKq

Rethinking Memory System Design Lecture @stanfordonline :
https://www.youtube.com/watch?v=F7xZLNMIY1E&list=PL5Q2soXY2Zi8D_5MGV6EnXEJHnV2YFBJl&index=4

← Lecture 18: SIMD Architectures · Lecture 19: GPU Architectures →