LESSON 6.1 · Systems · 90 min
Prefill and decode: two different workloads
Prefill and decode: two different workloads. Learn compute-bound versus memory-bound work, latency, and throughput and complete: Profile one complete generation. Part of the “Make
Learning objectives
- Explain what problem “Prefill and decode: two different workloads” solves without hiding behind terminology.
- Trace the variables and causal links across compute-bound versus memory-bound work, latency, and throughput.
- Complete “Profile one complete generation” and judge the result with evidence rather than intuition.
Core concepts
compute-bound versus memory-bound work
compute-bound versus memory-bound work is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
latency
latency is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and throughput
and throughput is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Profile one complete generation
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.