Tag
In an artificial neuron, weights measure how important each input is and the bias shifts the result. The neuron multiplies each input by its weight, adds everything up and includes the bias to produce the weighted sum z = Wx + b, the number that then passes through the activation function.
The perceptron is the simplest artificial neuron: it takes several inputs, multiplies them by its weights, adds a bias and applies an activation function that decides between two outputs. Frank Rosenblatt introduced it in 1958, and it remains the basic building block of every modern neural network.
Probability in neural networks rests on three ideas: a distribution assigns weights to possible outcomes, the expected value averages those outcomes, and likelihood measures how well the model fits the data. From those three pieces come the softmax function and the cross-entropy loss that we optimise.
The chain rule computes the derivative of a composite function by multiplying the derivatives of its links: if y depends on u and u depends on x, then dy/dx equals dy/du times du/dx. That layer-by-layer multiplication of derivatives is exactly what backpropagation does to train a neural network.
The essential differentiation rules are a handful of formulas that turn any function into its derivative: the power rule, the product and quotient rules, and the rules for the exponential and the logarithm. With them, plus the chain rule, a neural network computes gradients and learns by adjusting its weights.
A derivative measures the rate of change of a function: how much its output varies when the input changes a little. In a neural network, that slope tells us in which direction and how strongly to adjust each weight to reduce the error, and it is the foundation of gradient descent and backpropagation.
The transpose swaps rows for columns and shows up at every step of backpropagation; the identity matrix acts as the 1 of matrix algebra and leaves any vector unchanged, and the inverse matrix undoes a transformation, though it only exists when the matrix is square and its determinant is not zero. Three operations that hold up the maths of a network.
Matrix multiplication is the core operation of a neural network: each layer gathers its weights into a matrix W and computes its output as the product W times X. That single operation, repeated layer after layer, turns the inputs into predictions and explains why graphics cards dominate modern deep learning.
The dot product multiplies each input by its weight and adds the results into a single number. A neuron uses that operation to compute its weighted sum z equals w times x plus the bias b, and that value decides, after the activation, how strongly the neuron fires in response to the data it receives.
The mathematics of neural networks rests on three blocks: linear algebra represents data and weights as vectors and matrices, calculus with derivatives and the chain rule lets the network learn through gradient descent, and probability shapes the loss functions. This roadmap walks that path from beginning to end so you know what to study and in what order.
