Batch, Stochastic and Mini-Batch Gradient Descent
About this video
Batch, stochastic and mini-batch: three ways to traverse the data.
Batch gradient descent uses all the data at each step, the stochastic version uses a single sample and mini-batch picks an intermediate group, usually 32 to 256 examples. This guide compares the three variants, their cost, their noise and why mini-batch has become the standard for training modern networks.
Video transcript
- 0:01 Full-batch descent uses all the data on every step:
- 0:04 a smooth path, but each step is expensive.
- 0:08 SGD uses a single sample and moves with a lot of noise.
- 0:12 Mini-batch sits in between.
More videos
Explainer
Firecrawl: Web Data for Agents
If you write your own crawler: robots.txt, one queue per host and 304 Not Modified make the polite option the cheap one too.
Explainer
How to Install SearXNG with Docker
One use for your own instance: an AI agent that finds its own sources by querying SearXNG, without paying per search.
Explainer
The Computational Graph and Automatic Differentiation
The computational graph and automatic differentiation, in action.
Explainer
Backpropagation: The Step-by-Step Derivation
The backpropagation derivation, factor by factor with the chain rule.