Build the mental model
Backpropagation is the specific algorithm that makes `.backward()` efficient: rather than the naive alternative — nudge one parameter slightly, re-run the entire forward pass, see how much the loss changed, repeat for every single parameter one at a time — backpropagation computes the gradient of the loss with respect to every parameter in the network in a single backward sweep. It does this by applying the chain rule systematically from the output backward to the input, reusing intermediate results at each layer instead of recomputing them, which is what makes training networks with millions of parameters computationally feasible at all; the naive approach would require one full forward pass per parameter, which for a million-parameter network is a million times slower. A training loop assembles this into repetition: for each epoch (one full pass over the dataset), for each batch, do a forward pass to get predictions, compute the loss, call loss.backward() to backpropagate, step the optimizer to update parameters, then zero the gradients — repeating until the loss stops meaningfully decreasing, which is what 'the model has learned' concretely means.
Connect it to a real scenario
This complete loop is exactly what trains every real feature on the Tutorial Platform: the sentiment classifier learns from thousands of past feedback comments, the ranking model learns from learner click patterns, and the recommendation model learns from lesson-completion sequences — all by running this same forward -> loss -> backward -> step -> zero_grad cycle over their respective datasets until loss converges. Everything covered in this chapter (tensors, autograd, nn.Module, loss functions, optimizers) exists to make this one loop possible; the more advanced architectures in later chapters change what happens inside the forward pass, but the training loop itself stays this same shape.
Try the working example
import torch
import torch.nn as nn
torch.manual_seed(0)
# Synthetic data: y = 2x + 1, plus a little noise
x = torch.linspace(-5, 5, 100).unsqueeze(1) # shape (100, 1)
y = 2 * x + 1 + torch.randn(x.shape) * 0.5
model = nn.Linear(1, 1)
loss_fn = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
for epoch in range(200):
predictions = model(x) # forward pass
loss = loss_fn(predictions, y) # compute loss
optimizer.zero_grad() # clear old gradients
loss.backward() # backpropagation
optimizer.step() # update parameters
if epoch % 40 == 0:
print(f"Epoch {epoch}: loss = {loss.item():.4f}")
learned_w = model.weight.item()
learned_b = model.bias.item()
print(f"Learned: y = {learned_w:.2f}x + {learned_b:.2f}")Prints the loss at epochs 0, 40, 80, 120, 160, decreasing steadily and leveling off near the irreducible noise variance (about 0.25, from the noise's standard deviation of 0.5), then prints the learned weight and bias, which converge close to 2.00 and 1.00 — the true values used to generate the synthetic data.5-minute try-it
Change the learning rate to 0.001 and re-run for 200 epochs — observe that the loss decreases much more slowly and the learned weight/bias are further from 2.0/1.0, then try 0.5 and see the training destabilize instead of converge.
One important caution
Putting optimizer.zero_grad() after loss.backward() instead of before it — the order in this lesson's code (zero_grad, then backward, then step) works, but many learners copy examples inconsistently and end up zeroing gradients right after computing them, which silently makes every step a no-op update.
Setting the learning rate too high in the hope of 'training faster' — for this simple linear regression, a learning rate much above ~0.12 causes the loss to oscillate or diverge instead of decrease, because the gradient step overshoots the minimum instead of approaching it.
Wikipedia — Backpropagation — Deep Learning