Modern CNN Architectures: ResNet

Build the mental model

As convolutional networks get deeper, you would expect accuracy to keep improving — more layers should mean more representational power. In practice, researchers found the opposite: past a certain depth, training accuracy actually gets worse, not because of overfitting (the test accuracy is worse too) but because very deep plain networks become genuinely harder to optimize. This is called the degradation problem. Intuitively, a deeper network should be able to at least match a shallower one — the extra layers could just learn the identity function and pass the input through unchanged. But it turns out that learning an identity mapping through a stack of nonlinear convolution-ReLU layers is surprisingly hard; gradients have to travel through many multiplications and nonlinearities, and small numerical drift compounds until the optimizer simply cannot find that identity solution reliably.

ResNet's fix is to stop asking each block to learn the full desired mapping H(x), and instead have it learn only the residual F(x) = H(x) - x, adding the original input back via a skip connection: output = F(x) + x. Now the 'do nothing' solution is trivial — the block just needs to push its weights toward zero, which gradient descent is very good at. More importantly, the skip connection gives gradients a direct, unimpeded path back to earlier layers during backpropagation: the derivative of the x term is exactly 1, so no matter how small the gradient through the convolutional path becomes, at least a full-strength gradient signal still reaches earlier layers. Think of it like an express elevator alongside a slow staircase — even if the staircase (the conv layers) gets crowded, there's always a fast route straight down.

Connect it to a real scenario

On the Tutorial Platform, a ResNet-style backbone is a natural fit for the profile-avatar moderation pipeline: because residual blocks let you train a genuinely deep feature extractor without the degradation problem crippling it, the platform can pull rich, discriminative embeddings from every uploaded avatar image and compare them against a bank of known-flagged images (nudity, copyrighted logos, spam patterns) using nearest-neighbor similarity — catching near-duplicates and slight edits that a shallow CNN's coarser features would miss.

• Very deep plain CNNs suffer from a degradation problem where training accuracy gets worse, not from overfitting but from optimization difficulty.
• A residual block learns F(x) = H(x) - x and adds the input back via a skip connection, making the identity mapping trivial to reach.
• Skip connections give gradients a direct path back to earlier layers, since the derivative of the skip term is always 1.

Try the working example

import torch
import torch.nn as nn
import torch.nn.functional as F

class ResidualBlock(nn.Module):
    def __init__(self, channels):
        super().__init__()
        self.conv1 = nn.Conv2d(channels, channels, kernel_size=3, padding=1)
        self.conv2 = nn.Conv2d(channels, channels, kernel_size=3, padding=1)

    def forward(self, x):
        out = F.relu(self.conv1(x))
        out = self.conv2(out)
        out = F.relu(out + x)
        return out

torch.manual_seed(0)
block = ResidualBlock(channels=16)
images = torch.randn(4, 16, 32, 32)  # fake batch of 4 feature maps
output = block(images)
print(output.shape)

torch.Size([4, 16, 32, 32]) — the residual block preserves both the number of channels (16) and the spatial dimensions (32x32) of the input, since both convolutions use padding=1 to keep the feature map size unchanged and the skip connection requires the shapes to match exactly for the addition to work.

Wikipedia — Residual neural network — Computer Vision