LESSON 4.8 · Theory · 100 min
Scaling laws and Chinchilla intuition
Scaling laws and Chinchilla intuition. Learn compute optimality, data/parameter ratios, and extrapolation risk and complete: Fit a scaling curve to small-model runs. Part of the “T
Learning objectives
- Explain what problem “Scaling laws and Chinchilla intuition” solves without hiding behind terminology.
- Trace the variables and causal links across compute optimality, data/parameter ratios, and extrapolation risk.
- Complete “Fit a scaling curve to small-model runs” and judge the result with evidence rather than intuition.
Core concepts
compute optimality
compute optimality is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
data/parameter ratios
data/parameter ratios is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and extrapolation risk
and extrapolation risk is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Fit a scaling curve to small-model runs
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.