LESSON 7.7 · Code Study · 150 min
nanochat: understand a complete chat system
nanochat: understand a complete chat system. Learn tokenizer, pretraining, midtraining, SFT, RL, and UI and complete: Draw the repository’s end-to-end dependency graph. Part of the
Learning objectives
- Explain what problem “nanochat: understand a complete chat system” solves without hiding behind terminology.
- Trace the variables and causal links across tokenizer, pretraining, midtraining.
- Complete “Draw the repository’s end-to-end dependency graph” and judge the result with evidence rather than intuition.
Core concepts
tokenizer
tokenizer is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
pretraining
pretraining is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
midtraining
midtraining is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
SFT
SFT is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
RL
RL is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Draw the repository’s end-to-end dependency graph
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.