The Softmax function transforms a neural network's output logits into a probability distribution that sums to 1. It is the standard activation for multi-class classification, from image classifiers to the vocabulary layer of language models like GPT.
The linear function f(x) = ax + b is the simplest activation a neural network can use: in its identity form, f(x) = x, it is the standard choice for the output layer in regression, because it does not bound the range of possible values. In hidden layers it fails, because composing several linear functions collapses the whole network into one equivalent linear layer.
6 min1994.4
We use first- and third-party cookies to analyze site traffic. You can accept them, reject them, or configure your choice.
Learn more about cookies
Cookie preferences
NecessaryEssential for the site to work. Always on.
AnalyticsHelp us understand how the site is used (Google Analytics).
