Technology
What Is an Activation Function and Why It Is Needed
An activation function is the nonlinear operation each neuron applies to its weighted sum z to produce its output a equals f of z. Without it, stacking layers only chains linear transformations and the whole network collapses into a single one. That nonlinearity is what lets a network learn complex patterns.
