Definition
Backpropagation computes the gradient of a scalar loss with respect to every trainable parameter that affected it. A forward pass records the intermediate values needed by the graph. The backward pass starts at the loss and applies the chain rule through each operation in reverse order, accumulating each parameter's contribution.
The method reuses intermediate derivatives, which is why it can compute many parameter gradients in roughly the cost of a small number of forward evaluations rather than perturbing every weight separately. Automatic-differentiation systems implement this pattern as reverse-mode differentiation.
The 1986 account
David Rumelhart, Geoffrey Hinton, and Ronald Williams gave back-propagation its best-known neural-network treatment in a 1986 Nature paper. Their procedure adjusted connection weights to reduce output error and showed that hidden units could learn task-relevant features. The paper popularized backpropagation for multilayer neural networks; the chain rule it uses is older mathematics.
Distinguish it from nearby terms
Backpropagation computes gradients. An optimizer such as stochastic gradient descent or Adam decides how to use them to update parameters. The loss function defines what is being minimized, while backpropagation carries that loss signal through the graph.
Check your understanding
A model's loss is correct, and backpropagation returns gradients, but the weights never change. Which part is missing: the forward pass, the backward pass, or the optimizer step? Explain the boundary.