Tag
A partial derivative measures how a function changes when you move just one of its variables and hold the rest fixed. The gradient gathers all those partial derivatives into a vector that points toward the steepest ascent; in a neural network, moving the opposite way lowers the error and drives the training.
The chain rule computes the derivative of a composite function by multiplying the derivatives of its links: if y depends on u and u depends on x, then dy/dx equals dy/du times du/dx. That layer-by-layer multiplication of derivatives is exactly what backpropagation does to train a neural network.
The essential differentiation rules are a handful of formulas that turn any function into its derivative: the power rule, the product and quotient rules, and the rules for the exponential and the logarithm. With them, plus the chain rule, a neural network computes gradients and learns by adjusting its weights.
A derivative measures the rate of change of a function: how much its output varies when the input changes a little. In a neural network, that slope tells us in which direction and how strongly to adjust each weight to reduce the error, and it is the foundation of gradient descent and backpropagation.
