LESSON 4.4 · Diagnosis · 110 min
Mixed precision and numerical stability
Mixed precision and numerical stability. Learn FP32, FP16, BF16, loss scaling, and overflow and complete: Reproduce and fix NaN training. Part of the “Turn Training into a System”
Learning objectives
- Explain what problem “Mixed precision and numerical stability” solves without hiding behind terminology.
- Trace the variables and causal links across FP32, FP16, BF16.
- Complete “Reproduce and fix NaN training” and judge the result with evidence rather than intuition.
Core concepts
FP32
FP32 is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
FP16
FP16 is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
BF16
BF16 is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
loss scaling
loss scaling is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and overflow
and overflow is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Reproduce and fix NaN training
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.