Binary Cross-Entropy

Binary cross-entropy is the standard loss function for two-class classification. It compares the probability returned by the sigmoid with the true label, 0 or 1, and punishes confident mistakes hard. Its formula comes from maximum likelihood, and its derivative combines with the sigmoid into a very simple gradient.

July 16, 2026 8 min 4

Artificial Intelligence

The Sigmoid Function: A Key Tool in Neural Networks

The sigmoid function maps any real number to a value between 0 and 1, which makes it the natural activation function for expressing probabilities in a neural network. It is differentiable across its whole domain, though it suffers from saturation and vanishing gradients in deep networks, so today it is mostly reserved for the output layer.

March 24, 2023 4 min 797 4.2