Lecture 7: Category Selectivity, Controversies, and MVPA

What this lecture covers

The lecture finishes a design discussion (within-subject designs, localizer scans, block vs. event-related designs, factorial designs) and then turns critical on the course's central claim so far: that certain brain regions are selective for faces, places, and bodies. It walks through the main lines of pushback on that claim and covers, in detail, Jim Haxby's argument that a region's mean response can hide information in the pattern of its response, which leads into multi-voxel pattern analysis (MVPA) and neural decoding.

After watching, you can explain main effects and interactions in a 2x2 fMRI design, describe why localizer scans are done within each subject, state the strongest empirical challenge to category-selective regions, and explain how a neural decoder is trained and tested to read out what stimulus produced a given pattern of brain activity.

Key ideas

• Within-subject and within-run designs: conditions should be distributed across subjects and within scanning runs, not assigned to different groups, because individual differences (in brain responsiveness, alertness, etc.) would confound the comparison.
• Subject-specific localizers: because functional regions vary in exact location across people (like faces vary in feature position), a candidate region such as the FFA must be located individually in each subject rather than assumed from a group average.
• Block vs. event-related designs: grouping trials of one condition together (block) versus interleaving them (event-related) trades off adaptation/anticipation biases against the difficulty of separating strongly overlapping, slow BOLD responses; overlapping responses can still be mathematically decomposed because they sum approximately linearly.
• Main effect vs. interaction: a main effect is the overall difference between levels of one factor (e.g., faces vs. objects, averaged over attention); an interaction is whether the effect of one factor depends on the level of another (e.g., whether face selectivity changes with attention).
• Replicated category-selective regions: across many stimulus categories tested, only faces, places (scenes), and bodies reliably produce selective regions across subjects; categories like tools, hands, flowers, or snakes have not shown consistent, replicable selective patches.
• Haxby's challenge: even when a region's mean response to a category is low, the pattern of response across voxels can still carry information distinguishing that category from another, which can be tested by checking whether patterns are more correlated within a category (e.g., chairs vs. chairs) than between categories (e.g., chairs vs. cars).
• MVPA (multi-voxel pattern analysis): analyzing patterns of activity across many voxels, rather than a single mean response, to detect information a region holds about a stimulus.
• Neural decoding: training a classifier on brain response patterns to known stimuli, then testing it on an unknown pattern to infer what stimulus produced it; this works far better with individual-neuron recordings in monkeys than with fMRI, because each fMRI voxel averages over hundreds of thousands of neurons.

Walkthrough

Wrapping up experimental design (0:10)

The lecture reviews minimal pairs, avoiding mismatched tasks across conditions (a confound), and why baseline conditions matter for computing selectivity as a ratio rather than just a raw difference. It then covers within-subject designs (using an analogy about unfairly assigning different graders to different students) and the same logic for allocating conditions within scanning runs.

Block vs. event-related designs and subject-specific localizers (5:15)

The lecture contrasts blocked designs (which risk adaptation and anticipation effects) with event-related designs (which risk overlapping, hard-to-separate BOLD responses), noting that overlapping responses can be recovered because they sum approximately linearly. It then explains why regions like the fusiform face area must be localized separately in each subject, since brains differ anatomically and functionally as much as faces do, using an analogy about locating skin lesions on individual photographs rather than a group average.

Factorial designs, main effects, and interactions (14:21)

Using a 2x2 design that crosses stimulus type (faces vs. objects) with task (attending to the stimulus vs. a demanding letter-monitoring task), the lecture defines main effect (an overall difference along one factor) and interaction (whether the effect of one factor depends on the level of another). Student volunteers draw out different possible data patterns at the board to illustrate cases with no effects, a main effect only, two main effects with no interaction, and a genuine interaction, clarifying that interaction lines need not visually cross.

Category-selective regions and where they replicate (33:39)

The lecture describes an experiment scanning subjects on around 20 object categories (including snakes, spiders, food, tools, and flowers) to search for additional selective regions beyond faces, places, and bodies. Only faces, places, and bodies produced selectivity that replicated reliably across subjects; other candidate categories, including tools and hands, remain disputed or unreplicated.

Controversies over category selectivity (37:43)

Several lines of pushback are introduced: that selective regions have messy, non-discrete boundaries in unsmoothed data; that they may be peaks in a broader continuous response landscape rather than distinct "things"; and that selectivity might be explainable by low-level perceptual features (such as curvature) rather than category per se.

Haxby's pattern-information challenge (41:45)

The lecture works through Jim Haxby's method in detail: even if the FFA's mean response to chairs and cars is low, the pattern of response across its voxels can still be more similar within a category (chairs to chairs) than between categories (chairs to cars), which would mean the region carries information about non-face objects. It traces the resulting back-and-forth in the literature, ending with the current consensus that such discrimination is real but weak, and offers the analogy of a vending machine that can also tell time: a system built for one purpose can still carry decodable information about something else.

Neural decoding and MVPA (56:00)

The lecture generalizes Haxby's method into neural decoding: train a decoder on brain patterns from known stimuli, then test whether an unknown pattern is more similar to one trained category than another. It covers MVPA terminology, decoding from whole-brain data versus a specific region of interest, decoding from MEG at successive time points to trace the time course of information processing, and a comparison study showing that decoding face identity from monkey single-neuron recordings works far better than from fMRI in the same brain region, because each voxel averages over huge numbers of neurons.

Testing how abstract a representation is (1:07:07)

The lecture closes by describing how training a decoder on one set of stimulus conditions (e.g., one viewpoint and color of a shoe) and testing it on a different set (a different viewpoint and color) can reveal whether a region's representation is abstract and invariant, rather than tied to superficial visual features.

Before you watch

• Review the previous lecture's introduction to independent/dependent variables, confounds, and minimal pairs, since this lecture builds directly on that vocabulary.
• Be familiar with the fusiform face area, parahippocampal place area, and body-selective regions established in earlier lectures.
• Basic comfort with the idea of correlation will help with the Haxby decoding method.

Check your understanding

1. Why must a candidate face-selective region be localized separately in each subject rather than using a group-averaged location?
2. In a 2x2 design crossing stimulus type and attention, what specifically would an interaction between these two factors tell you that a main effect of stimulus type alone would not?
3. How does Haxby's method test whether a region carries information about a category even when its average response to that category is low?
4. Why does neural decoding work much better with monkey single-neuron recordings than with fMRI in the same brain region?
5. What does it mean, and why does it matter, if a decoder trained on one viewpoint of an object successfully classifies a different viewpoint of the same object?