CNNs for Vision Recap

Build the mental model

You've already built and trained CNNs in the Deep Learning with PyTorch course, so this is a fast recap focused specifically on why Conv2d and MaxPool2d are the right tools for images rather than a from-scratch explanation of either. The key property is weight sharing: a single small filter (say, 3x3x3 for an RGB input) is reused at every spatial position across the whole image, rather than a fully-connected layer learning an entirely separate weight for every pixel position. This has two consequences that matter specifically for vision: first, it's dramatically cheaper — a 3x3 conv filter over a 32x32 image has the same tiny parameter count whether the image is 32x32 or 3200x3200, while a dense layer's parameters would explode with image size. Second, and more importantly, it gives the network translation equivariance: a pattern the filter learns to detect (an edge, a texture) gets detected wherever it appears in the image, because the exact same weights slide across every location — the network doesn't have to separately relearn 'edge' for the top-left corner and the bottom-right corner.

Pooling (MaxPool2d here) complements weight sharing by adding a controlled amount of local translation invariance on top of equivariance: taking the max over each small window means that shifting the input by a pixel or two often leaves the pooled output unchanged, since the same strongest activation is likely still inside the window. Stacking conv+pool blocks also grows the effective receptive field cheaply — each pooled layer lets the next convolution's small kernel 'see' a proportionally larger region of the original image, so deep stacks build up from detecting small local patterns (edges) to detecting larger structures (shapes, then object parts) without ever needing a single giant kernel. Contrast this with a plain MLP, which flattens the image into a 1D vector and throws away all 2D spatial structure — it has no notion that two nearby pixels are related, which is exactly why CNNs generalize far better than MLPs on the same amount of image data.

Connect it to a real scenario

The Tutorial Platform's auto-tagging feature for uploaded lesson images (labeling each upload as 'diagram', 'code screenshot', or 'photo' before it's inserted into a lesson) runs exactly this kind of small Conv2d+MaxPool2d feature extractor as its first stage — weight sharing is what lets the same 'monospace text block' detector fire whether the code screenshot was pasted in the top-left or bottom-right of the uploaded image, without needing separately-trained detectors for every possible position.

• Weight sharing means the same small filter is applied at every spatial location, giving translation equivariance and far fewer parameters than a dense layer over the same input.
• Max pooling adds local translation invariance and cheaply grows the effective receptive field as conv+pool blocks are stacked.
• A plain MLP flattens away 2D spatial structure entirely; that's the core reason it generalizes worse than a CNN on the same amount of image data.

Try the working example

import torch
import torch.nn as nn

torch.manual_seed(0)

# Fake batch of 4 RGB "lesson images" the model would see during
# auto-tagging: diagram vs code-screenshot vs photo classification.
images = torch.randn(4, 3, 32, 32)

feature_extractor = nn.Sequential(
    nn.Conv2d(in_channels=3, out_channels=8, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(kernel_size=2),   # 32x32 -> 16x16
    nn.Conv2d(in_channels=8, out_channels=16, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(kernel_size=2),   # 16x16 -> 8x8
)

features = feature_extractor(images)
print("input shape:", images.shape)
print("feature map shape:", features.shape)

# The same 3x3 conv weights are reused at every spatial position --
# count the parameters to see how cheap that is compared to a dense
# layer over the same input.
conv1_params = sum(p.numel() for p in feature_extractor[0].parameters())
print("params in first conv layer:", conv1_params)

Prints input shape: torch.Size([4, 3, 32, 32]), then feature map shape: torch.Size([4, 16, 8, 8]) — each of the two MaxPool2d(kernel_size=2) layers halves H and W (32 → 16 → 8), while the channel count follows the Conv2d layers (3 → 8 → 16). Finally prints params in first conv layer: 224, computed exactly as (8 output channels × 3 input channels × 3 × 3 weights) + 8 biases = 216 + 8.

Wikipedia — Convolutional neural network — Computer Vision