LESSON 2.4 · Lab · 120 min
MLP language models and data splits
MLP language models and data splits. Learn Bengio language models, train/dev/test, underfitting, and overfitting and complete: Reproduce the makemore MLP. Part of the “Language Bec
Learning objectives
- Explain what problem “MLP language models and data splits” solves without hiding behind terminology.
- Trace the variables and causal links across Bengio language models, train/dev/test, underfitting.
- Complete “Reproduce the makemore MLP” and judge the result with evidence rather than intuition.
Core concepts
Bengio language models
Bengio language models is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
train/dev/test
train/dev/test is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
underfitting
underfitting is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and overfitting
and overfitting is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Reproduce the makemore MLP
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.