Categorical Cross-Entropy

Categorical cross-entropy is the standard loss function for classifying into several mutually exclusive classes. It compares the distribution the network predicts, usually after a softmax, with the true label written in one-hot format, and it penalises heavily any low probability assigned to the correct class.

July 16, 2026 7 min

Artificial Intelligence

Softmax Function: Activation for Classification

The Softmax function converts a vector of logits (arbitrary values) into a probability distribution where every value is positive and the values sum to exactly 1. It is the standard output-layer activation for multi-class classification, and the final operation language models use to predict the next token.

March 24, 2023 4 min 590 4.1