Python Tutorial (CS224N Review Session)

What this lecture covers

This is a teaching-assistant review session, not a graded lecture, aimed at bringing everyone up to speed on Python and NumPy before the course's second homework, which requires implementing a feedforward network directly in NumPy. It assumes no prior Python experience while still moving quickly enough to be useful for students who already know the language.

The session works through core Python syntax and data structures (variables, lists, tuples, dictionaries, loops), then spends the second half on NumPy: how arrays and their shapes are represented, indexing and slicing, matrix multiplication versus the dot product, and broadcasting. After watching, you should be able to read and write basic Python code, choose between lists, tuples, and dictionaries appropriately, and reason about NumPy array shapes well enough to build and debug matrix operations for the homework.

Key ideas

• Why Python for NLP: Python is high-level and readable, has strong scientific-computing support, and libraries such as PyTorch and TensorFlow interface directly with it, which is why it dominates deep learning work.
• Dynamic typing: a Python variable is not bound to one type; the same name can be reassigned from an integer to a string without error.
• Lists vs. tuples: lists are mutable (you can append, delete, and change elements) while tuples are immutable once created; both can hold mixed types, including other lists.
• Dictionaries: a dictionary maps keys to values for fast lookup, commonly used in NLP to map strings (words) to numeric indices.
• NumPy arrays vs. lists: NumPy arrays are backed by optimized C/C++ subroutines, so mathematical operations on them run far faster than equivalent Python loops, but NumPy functions only work on NumPy arrays, not plain lists.
• Shape matters: a shape like (3,) is a flat list of 3 values, (1, 3) is one row of 3 columns, and (3, 1) is three rows of one column each; getting shapes right is essential for matrix operations.
• Matrix multiplication vs. dot product: np.matmul (or @) multiplies matrices under the rule that the inner dimensions must match, while np.dot behaves like a true dot product only for 1-D vectors and otherwise falls back to matrix multiplication.
• Broadcasting: NumPy can apply an operation between arrays of different shapes by duplicating the smaller array along any dimension where its size is 1, without you writing an explicit loop.

Walkthrough

Why Python, and language basics (0:05)

The instructor explains why Python is the standard language for NLP and deep learning, then covers language fundamentals: dynamic typing, arithmetic and exponentiation, type casting between floats, integers, and strings, boolean values (capitalized True/False), Python's None in place of null, and the use of indentation instead of braces to delimit code blocks.

Core data structures: lists, tuples, and dictionaries (6:10)

Working in a Colab notebook, the instructor demonstrates zero-indexed lists, appending elements, concatenating lists with +, and slicing syntax where the start index is included and the end index is excluded, including shorthand slices and negative indexing from the end of a list. Tuples are introduced as immutable equivalents of lists, and dictionaries as key-to-value mappings useful for converting strings to numeric indices, with membership checks and the del keyword for removing entries.

Loops over lists and dictionaries (14:13)

The lecture covers iterating with range(), iterating directly over list elements, and using enumerate() to get both an element and its index at once. For dictionaries, it distinguishes iterating over keys directly, over .values(), and over both with .items().

Introducing NumPy and array shapes (17:16)

NumPy is introduced as a library backed by fast C/C++ subroutines for matrix and vector math, distinct from Python lists and only usable on arrays created with np.array(). The instructor explains the conventional distinction between vectors (one dimension), matrices (two dimensions), and tensors (more than two dimensions, often used with PyTorch on GPUs), and works through how shapes like (1, 3) and (3, 1) differ, since this distinction matters later for broadcasting.

Array operations: max, multiplication, and the dot product (22:18)

The session covers reshape() for changing an array's dimensions without changing its data, the axis argument for functions like np.max (axis 0 reduces over rows, axis 1 over columns), and keepdims for preserving the original number of dimensions in a result. It then distinguishes element-wise multiplication (*, requiring matching shapes) from true matrix multiplication (np.matmul or @, requiring the inner dimensions to match), and shows that np.dot gives a scalar dot product only for 1-D vectors, behaving as matrix multiplication otherwise.

Indexing and broadcasting (37:28)

The instructor demonstrates selecting specific rows and columns with combinations of indices and :, boolean indexing (for example, selecting all values above a threshold), and np.newaxis for adding a dimension. Broadcasting is presented as NumPy's mechanism for applying operations between arrays of different but compatible shapes: two dimensions are compatible if they are equal or if one of them is 1, in which case NumPy duplicates the size-1 dimension internally rather than requiring an explicit copy, which is faster and is how the same weight matrix gets applied across a batch of inputs.

Preferring NumPy operations over explicit loops (45:33)

The lecture closes with a concrete comparison: adding a value to a slice of a large matrix using a Python loop versus using NumPy's arange and vectorized addition, noting that loops over large arrays are typically around a hundred times slower than the equivalent NumPy operation.

Before you watch

• No prior Python experience is required, though basic programming familiarity (variables, functions) makes the pace easier.
• Having the course's second homework assignment open alongside the video helps connect the NumPy operations to what you will actually implement.

Check your understanding

1. What is the practical difference between a Python list and a tuple, and when would you choose one over the other?
2. Why do NumPy operations run much faster than the equivalent Python for loops on large arrays?
3. Given arrays of shape (3, 4) and (3, 1), explain why they are broadcast-compatible and what happens during an element-wise addition between them.
4. How does np.dot behave differently on two 1-D vectors versus two 2-D matrices?
5. Why does the axis argument matter when calling a reduction function like np.max, and what does axis=0 versus axis=1 mean for a 2-D array?