Build the mental model
A good keypoint — the kind classical detectors like Harris corners or SIFT try to find — is a location where the local image content is distinctive enough to re-identify reliably, and the standard way to formalize 'distinctive' is by asking what happens to the pixel values under a small window if you shift that window slightly in any direction. In a flat, uniform region, shifting the window barely changes anything — there's nothing distinctive there. Along a straight edge, shifting the window parallel to the edge barely changes anything either, even though shifting perpendicular to it changes a lot — an edge is only distinctive in one direction, which is why edges make poor keypoints even though they have strong gradients. At a true corner, shifting the window in any direction changes the content substantially, because two edges meet there — that two-directions property is what a real corner detector (like Harris, via the structure tensor) explicitly tests for, going one step beyond the simple single-direction gradient magnitude this lesson's code computes.
Classical feature detectors like SIFT and Harris corners encode a human's hypothesis about what 'distinctive and re-findable' means, expressed as an explicit mathematical test anyone can read, reason about, and run without training data. A CNN has no such explicit hypothesis — nothing in its architecture says 'find corners.' Yet when researchers visualize what a trained CNN's early convolutional filters actually respond to, they frequently resemble edge and blob detectors remarkably similar to hand-designed ones, learned purely because those patterns turned out to be useful for minimizing the training loss. Deeper layers go further, learning detectors for combinations of patterns — fur texture, wheel-like curves, eye-like blobs — that would be extremely difficult for a person to hand-design a mathematical rule for. This is why classical detectors remain useful for cheap, interpretable, training-free tasks like image stitching or simple tracking, while learned features dominate wherever the pattern of interest is too complex to describe by hand.
Connect it to a real scenario
The Tutorial Platform's course-thumbnail uploader runs a cheap classical fingerprint check before it ever calls the more expensive learned similarity model: it computes a gradient-magnitude 'keypoint density' map for every newly uploaded thumbnail (similar to this lesson's code) and compares it against existing thumbnails' fingerprints, catching the common case of an instructor re-uploading the exact same or a barely-cropped banner image almost instantly, and only falling back to the slower learned embedding model when the cheap classical check can't confidently rule duplication in or out.
Try the working example
import torch
import torch.nn.functional as F
torch.manual_seed(0)
# Synthetic grayscale image standing in for a course thumbnail.
image = torch.rand(1, 1, 20, 20)
sobel_x = torch.tensor([[-1., 0., 1.],
[-2., 0., 2.],
[-1., 0., 1.]]).view(1, 1, 3, 3)
sobel_y = torch.tensor([[-1., -2., -1.],
[ 0., 0., 0.],
[ 1., 2., 1.]]).view(1, 1, 3, 3)
grad_x = F.conv2d(image, sobel_x, padding=1)
grad_y = F.conv2d(image, sobel_y, padding=1)
gradient_magnitude = torch.sqrt(grad_x ** 2 + grad_y ** 2)
# A classical corner/keypoint detector is, at its core, looking for
# locations where this gradient magnitude is unusually high in
# *multiple* directions -- flat regions and simple edges score low,
# corner-like regions score high. We approximate that here with a
# blunt threshold instead of a full Harris response.
threshold = gradient_magnitude.mean() + gradient_magnitude.std()
keypoint_mask = gradient_magnitude > threshold
print("gradient_magnitude shape:", gradient_magnitude.shape)
print("keypoint_mask shape:", keypoint_mask.shape)
print("number of candidate keypoints:", keypoint_mask.sum().item())
print("fraction of image flagged as a keypoint:", (keypoint_mask.float().mean() > 0).item())Prints gradient_magnitude shape: torch.Size([1, 1, 20, 20]) and keypoint_mask shape: torch.Size([1, 1, 20, 20]). number of candidate keypoints is a small positive integer well under the total of 400 pixels — the threshold (mean + one standard deviation) by construction only keeps the highest-magnitude tail of the distribution, so only a modest fraction of pixels pass; the exact count is deterministic given the seed but not hand-traced here. The final line prints fraction of image flagged as a keypoint: True, since at least some pixels exceed the threshold by definition of mean + std on non-constant data.5-minute try-it
Replace the blunt mean+std threshold with a Harris-like response: compute Ixx = grad_x**2, Iyy = grad_y**2, Ixy = grad_x*grad_y, sum each over a small neighborhood (e.g. with a 3x3 average-pooling), then compute response = (Ixx*Iyy - Ixy**2) - 0.04*(Ixx+Iyy)**2 and threshold that instead — compare how many keypoints this flags versus the original gradient-magnitude approach on the same image.
One important caution
Treating a raw gradient-magnitude threshold (as in this demo) as a corner detector — it actually fires on straight edges just as strongly as on corners, since it never checks whether the gradient varies in more than one direction.
Assuming a keypoint found this way is scale- and rotation-invariant by default — plain gradient-based detection has no built-in invariance, so the same thumbnail resized or rotated will generally produce a different-looking keypoint map unless you deliberately add multi-scale or orientation handling, unlike a well-trained CNN embedding which tends to be far more robust to those changes automatically.
Wikipedia — Feature (computer vision) — Computer Vision