Episode 1: Overview of Machine Learning Compilation

What this lecture covers

This opening lecture asks why deploying a machine learning model is hard: a model built in a framework like PyTorch has to run on wildly different targets, from data-center GPUs to phones and embedded sensors, each with its own operating system, runtime libraries and accelerators. The lecture frames machine learning compilation (MLC) as the process of transforming a model from its development form into a deployment form that is minimal, integrates hardware-specific backends, and takes advantage of native acceleration.

As the first session of the course, it sets up the vocabulary the rest of the series builds on: development form versus deployment form, and the pair of concepts tensor and tensor function. After watching, you should be able to explain what problem MLC solves, list its three goals, and recognize why the same computation can be expressed in several different abstractions.

Key ideas

• Development form: the model as built in a framework such as PyTorch or TensorFlow, before any deployment-specific packaging.
• Deployment form: everything needed to run a model on a target device, including an execution engine, learned weights, supporting libraries, and the OS/hardware interfaces that connect them.
• Integration and dependency minimization: one MLC goal is combining backend components from different vendors while keeping the final deployment artifact as small as possible.
• Hardware-native acceleration: a second goal is generating deployables that use accelerators like tensor cores or TPUs rather than ignoring them.
• General optimization: a third goal covers reducing memory usage, improving execution efficiency, and scaling across multiple heterogeneous devices.
• Tensor: a multi-dimensional array holding the inputs, outputs and intermediate results of a model's computation.
• Tensor function: an operation, or composed sequence of operations, that transforms input tensors into output tensors, such as a linear layer or a fused linear-plus-activation block.
• Abstraction / implementation: the same tensor function can be represented at different levels of detail (a computational graph, a loop-level program, or low-level code); a more specialized representation is called an implementation of a more general abstraction.

Walkthrough

Why the AI software landscape needs MLC (0:42)

The lecture contrasts older general-purpose software, built on broad libraries for a single device class, with today's AI software, which must move specialized models onto a widening range of hardware: data-center CPUs, TPUs, and phones. This shift is what motivates treating deployment as its own engineering problem rather than an afterthought.

The machine learning deployment problem (3:44)

Using examples like self-driving cars, voice assistants and protein folding, the lecture shows that each application needs to reach a different deployment environment. It walks through what deploying a model onto an Android device actually requires: an execution engine, stored weights, supporting libraries for operations such as matrix multiplication, and a runtime such as OpenCL or a native acceleration API. MLC is defined here as the process that transforms a model from development form to deployment form, whether that transformation is done manually by engineers or partly automated.

The three MLC goals (13:47)

The lecture explains integration and dependency minimization (assembling backend components from different teams or vendors into a minimal deployable), leveraging hardware-native acceleration (targeting features like tensor cores), and general optimization (memory, execution efficiency, and scaling across devices). It notes these goals overlap rather than having strict boundaries, and draws an analogy to traditional compilers like GCC, while pointing out that MLC does not always involve code generation.

Tensors and tensor functions (27:35)

The lecture defines tensors as the multi-dimensional arrays storing a model's data, using a multilayer perceptron on flattened images as the running example. It then defines tensor functions as the operations connecting tensors, showing a linear projection as output[i,j] = sum_k(w[j,k] * x[i,k]), and demonstrates that a tensor function can represent either a single operator or a fused sequence of operators.

An example compilation process and abstractions (33:53)

Comparing a development-form graph (two separate linear and ReLU operations) to a deployment-form version (a single fused linear-ReLU implementation), the lecture shows why fusing operations can reduce memory copies and speed up GPU execution. It introduces abstraction and implementation as the key recurring concept: the same tensor computation can be shown as a graph, as fused operators, or as nested loops, each exposing a different level of detail.

Four abstraction categories and course logistics (41:57)

The lecture previews four abstraction categories the course will cover: computational graphs, tensor programs (loop-level code), library runtimes, and hardware primitives. It closes with course logistics: recordings on YouTube, code exercises on GitHub, lecture notes on mlc.ai, and a discussion forum for questions.

Before you watch

• Familiarity with a deep learning framework such as PyTorch, TensorFlow or JAX helps, since the examples assume you recognize layers like linear projections and ReLU.
• Basic Python and NumPy experience is expected, as later examples reference NumPy-style array computation.
• No prior systems or compiler background is required for this introductory episode.

Check your understanding

1. What is the difference between a model's development form and its deployment form?
2. What are the three goals of machine learning compilation described in the lecture?
3. Why might fusing a linear operation and a ReLU activation into one tensor function improve GPU performance?
4. What does it mean for two representations to be different abstractions of the same tensor function?
5. Name two of the four abstraction categories the course will cover in later episodes.