Artificial Intelligence
The Linear Activation Function
The linear activation function is f(x) = ax + b and, in its usual form, the identity f(x) = x: the neuron outputs its weighted sum untouched. Its derivative is constant and its range is unbounded, which makes it the output activation for regression. It fails in hidden layers, because two stacked linear layers equal a single one.