Tag
Backpropagation applies the chain rule to share a network's error layer by layer. You first compute the error of the output layer, propagate it backward, and use it to obtain the gradients of every weight and bias in a single pass, summarised in four compact equations you can code directly.
Backpropagation shares out the blame for the error among all the weights of a neural network. It propagates an error signal backwards layer by layer, multiplying by the local derivatives, and so obtains the gradient of every weight in a single pass. That idea, published in 1986, is what makes training deep networks possible.
A partial derivative measures how a function changes when you move just one of its variables and hold the rest fixed. The gradient gathers all those partial derivatives into a vector that points toward the steepest ascent; in a neural network, moving the opposite way lowers the error and drives the training.
