Categories

Technology

Mean Squared Error (MSE) as a Loss Function

Mean squared error (MSE) is the most common loss function in regression: it averages the square of the gap between each true value and its prediction. Squaring punishes big mistakes hard, keeps the function differentiable and gives gradient descent a clear, smooth target to minimise while training a model.

Artificial Intelligence

The Linear Activation Function

The linear activation function is f(x) = ax + b and, in its usual form, the identity f(x) = x: the neuron outputs its weighted sum untouched. Its derivative is constant and its range is unbounded, which makes it the output activation for regression. It fails in hidden layers, because two stacked linear layers equal a single one.