LESSON 1.6 · Lab · 110 min
Losses, regularization, and SGD
Losses, regularization, and SGD. Learn margin loss, batches, weight decay, and learning rate and complete: Train a two-moons classifier and plot its boundary. Part of the “Neural N
Learning objectives
- Explain what problem “Losses, regularization, and SGD” solves without hiding behind terminology.
- Trace the variables and causal links across margin loss, batches, weight decay.
- Complete “Train a two-moons classifier and plot its boundary” and judge the result with evidence rather than intuition.
Core concepts
margin loss
margin loss is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
batches
batches is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
weight decay
weight decay is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and learning rate
and learning rate is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Train a two-moons classifier and plot its boundary
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.