Hugging Face Tutorial (CS224N Review Session)

What this lecture covers

This is a teaching-assistant review session on the Hugging Face transformers and datasets libraries, aimed at students who will use pretrained Transformer models for a course project. It assumes familiarity with PyTorch from the earlier tutorial and focuses entirely on how to load, run, inspect, and fine-tune off-the-shelf NLP models rather than on the theory behind them.

The session moves from finding a model on the Hugging Face Hub, through tokenizers and pretrained model classes, to running inference and computing loss, inspecting a model's attention weights and hidden states, and finally fine-tuning a model on a small IMDb sentiment dataset using both a manual PyTorch training loop and Hugging Face's Trainer class. After watching, you should be able to load a pretrained model and its tokenizer, run inference, and fine-tune the model on your own labeled data.

Key ideas

• Model Hub: Hugging Face hosts pretrained weights for many architectures (BERT, GPT-2, T5, and others), searchable by task, that can be downloaded by name.
• Tokenizer + model pair: using a model requires a matching tokenizer, and AutoTokenizer / AutoModel automatically select the correct class for a given model name so you don't have to hardcode it.
• What a tokenizer produces: calling a tokenizer on a string returns input_ids (numeric token IDs) and an attention_mask, with options for padding and truncation to get fixed-length batches.
• Task-specific model classes: classes like AutoModelForSequenceClassification attach the right prediction head to a base architecture, and the architecture (encoder-only, decoder-only, or encoder-decoder) limits which tasks a given model can perform.
• Models are PyTorch modules: a loaded Hugging Face model behaves like any nn.Module, so .backward() and manual training loops work directly on it, and passing labels lets the model itself return a computed loss.
• Inspecting internals: setting output_attentions=True and output_hidden_states=True when loading a model exposes per-layer hidden states and per-head attention weights, useful for analyzing what a model attends to.
• datasets library: load_dataset and DatasetDict give convenient access to public datasets (such as IMDb) with a .map() method for batch preprocessing like tokenization.
• Two ways to fine-tune: you can write a standard PyTorch training loop by hand, or use Hugging Face's TrainingArguments and Trainer classes, which wrap the loop, evaluation, checkpointing, and callbacks like early stopping.

Walkthrough

Finding a model and understanding the two required pieces (0:04)

The instructor introduces the Hugging Face Hub as a source of freely downloadable pretrained models for tasks like sentiment analysis, zero-shot classification, and more. Using any model requires two components: a tokenizer, which converts raw text into vocabulary IDs, and the model itself, which produces predictions from those IDs; AutoTokenizer and AutoModelForSequenceClassification load the correct classes automatically from a model's name string.

How tokenizers work (5:07)

The lecture walks through calling a tokenizer directly on an input string to get input_ids and an attention_mask, accessible either as dictionary keys or as attributes. It covers the internal tokenization steps (splitting text into subword tokens, converting to IDs, adding special tokens), the difference between the Python and Rust-based "fast" tokenizers, and options for returning PyTorch tensors, padding to a fixed length, enabling truncation, and batch-decoding token IDs back into text.

Task-specific model classes and architecture constraints (14:14)

Different Hugging Face classes attach different heads to the same base architecture, such as DistilBertForSequenceClassification versus DistilBertForMaskedLM, and AutoModel picks the matching class automatically. The instructor distinguishes encoder-only models (BERT), decoder-only models (GPT-2), and encoder-decoder models (BART, T5), noting that an encoder-only model like DistilBERT cannot be used for sequence-to-sequence tasks that require a decoder.

Running inference and computing loss (18:19)

The lecture shows passing tokenizer outputs into a model either as explicit named arguments or by unpacking the dictionary with **model_inputs, and reading the resulting logits and predicted class. Because a loaded model is a regular PyTorch module, its parameters can be updated with .backward() and a standard loss function, but Hugging Face models also compute the loss internally when labels are passed in directly, and the predicted class can be read off with argmax on the logits.

Inspecting attention weights and hidden states (24:25)

Loading a model with output_attentions=True and output_hidden_states=True, then running it in eval() mode without tracking gradients, exposes per-layer hidden states and per-head attention weight matrices. The instructor demonstrates visualizing attention as a grid of heatmaps, one row per layer and one column per attention head, showing which tokens each head attends to.

Preparing a dataset for fine-tuning (31:35)

Using the datasets library, the lecture loads a small subset of the IMDb sentiment dataset, then uses .map() with a tokenization function to convert the raw text and labels into input_ids and an attention_mask in batches. It covers renaming the label column to labels, removing the now-unneeded text column, and setting the dataset format to PyTorch tensors so it can be wrapped in a standard DataLoader.

Fine-tuning: manual loop versus the Trainer class (38:39)

The instructor first shows a manual fine-tuning loop that mirrors ordinary PyTorch training, using an AdamW optimizer and a linear learning-rate schedule imported from transformers. As an alternative, TrainingArguments configures batch sizes, evaluation strategy, and learning rate, and the Trainer class wraps the entire loop, taking the model, arguments, datasets, tokenizer, and a metrics function; calling trainer.train() runs training, trainer.predict() runs evaluation, and checkpoints are saved automatically for later reloading. Optional callbacks, such as logging or early stopping, can be attached to the trainer.

Before you watch

• This tutorial assumes the PyTorch fundamentals covered in this course's PyTorch tutorial (tensors, nn.Module, training loops).
• Basic familiarity with what tokenization and a classification head are makes the tokenizer and model sections easier to follow.

Check your understanding

1. Why does using a pretrained model require its matching tokenizer rather than any tokenizer?
2. What is the difference between AutoModel, AutoModelForSequenceClassification, and a model-specific class like DistilBertForMaskedLM?
3. Why can't an encoder-only model like DistilBERT be used directly for a sequence-to-sequence task?
4. What do output_attentions=True and output_hidden_states=True add to a model's output, and why is model.eval() used when inspecting them?
5. What are the main differences between writing a manual PyTorch fine-tuning loop and using Hugging Face's Trainer class?