Technology
Batch, Stochastic and Mini-Batch Gradient Descent
Batch gradient descent uses all the data at each step, the stochastic version uses a single sample and mini-batch picks an intermediate group, usually 32 to 256 examples. This guide compares the three variants, their cost, their noise and why mini-batch has become the standard for training modern networks.