LESSON 1.2 · Derivation · 100 min
The chain rule is a message-passing rule
The chain rule is a message-passing rule. Learn local derivatives, upstream gradients, and gradient accumulation and complete: Annotate every edge of a computation graph with its g
Learning objectives
- Explain what problem “The chain rule is a message-passing rule” solves without hiding behind terminology.
- Trace the variables and causal links across local derivatives, upstream gradients, and gradient accumulation.
- Complete “Annotate every edge of a computation graph with its gradient” and judge the result with evidence rather than intuition.
Core concepts
local derivatives
local derivatives is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
upstream gradients
upstream gradients is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and gradient accumulation
and gradient accumulation is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Annotate every edge of a computation graph with its gradient
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.