Technology
Dropout and Its Mathematical Interpretation
Dropout is a regularization technique that switches neurons off at random during training, with retention probability p, and divides the surviving activations by p to preserve their scale. At inference the full network runs without dropping anything. Mathematically it amounts to averaging a huge ensemble of subnetworks that share weights.