The Computational Graph and Automatic Differentiation
About this video
The computational graph and automatic differentiation, in action.
A computational graph represents a function as a network of elementary operations. Automatic differentiation traverses that graph to compute exact derivatives: reverse mode, the basis of backpropagation, obtains the gradient of every weight in a single backward pass. It is the mechanism that makes training networks with billions of parameters possible.
Video transcript
- 0:01 A computational graph splits a calculation into operations:
- 0:05 multiply a by b, then add c.
- 0:08 Going backward, automatic differentiation applies the chain rule:
- 0:12 the derivative with respect to a is 2, and with respect to b, 3.
More videos
Explainer
Firecrawl: Web Data for Agents
If you write your own crawler: robots.txt, one queue per host and 304 Not Modified make the polite option the cheap one too.
Explainer
How to Install SearXNG with Docker
One use for your own instance: an AI agent that finds its own sources by querying SearXNG, without paying per search.
Explainer
Backpropagation: The Step-by-Step Derivation
The backpropagation derivation, factor by factor with the chain rule.
Explainer
Backpropagation: The Intuition
The intuition of backpropagation: the error flows backward through the network.