LESSON 1.4 · Lab · 80 min
Gradient checking: verify before trusting code
Gradient checking: verify before trusting code. Learn finite differences, central differences, and error scales and complete: Compare every operator against PyTorch. Part of the “N
Learning objectives
- Explain what problem “Gradient checking: verify before trusting code” solves without hiding behind terminology.
- Trace the variables and causal links across finite differences, central differences, and error scales.
- Complete “Compare every operator against PyTorch” and judge the result with evidence rather than intuition.
Core concepts
finite differences
finite differences is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
central differences
central differences is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and error scales
and error scales is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Compare every operator against PyTorch
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.