LESSON p.4 · Assessment · 180 min
Prerequisite gate: run your first training loop
Prerequisite gate: run your first training loop. Learn data, tensors, forward pass, loss, backward pass, updates, tests, and reproducibility and complete: Move from text counts to
Learning objectives
- Explain data → forward → loss → backward → update.
- Separate parameter values, gradients, and optimizer updates.
- Assert shapes, finiteness, and expected loss movement.
- Reproduce the experiment with a fixed seed and preserved failures.
Core concepts
Data and parameters
Inputs and labels are observations; parameters are state the optimizer changes. Similar shapes do not imply the same lifetime or requires_grad behavior.
Forward pass and logits
The forward pass maps inputs and parameters to unnormalized scores. Assert batch, class dimension, and finiteness before computing loss.
Scalar loss
Cross-entropy reduces batch predictions to an optimization scalar. A lower training loss is not evidence of generalization.
Backward and grad
Backward accumulates each leaf parameter’s derivative. Understand whether repeated calls intentionally accumulate before clearing gradients.
Parameter update
Update under no_grad, then clear gradients. The learning rate scales movement along the negative gradient; check both direction and new loss.
Build and verify
Move from text counts to one tested gradient update
- Fix two one-hot inputs, labels, initial weights, and seed; hand-check initial logits.
- Run forward, cross-entropy, backward, and one SGD update with a shape ledger.
- Assert finite loss, nonempty gradients, and changed parameters; recompute loss.
- Break label range, dtype, gradient clearing, and shape one at a time.