LESSON 4.5 · Systems · 140 min
GPUs, kernels, and the memory hierarchy
GPUs, kernels, and the memory hierarchy. Learn HBM, SRAM, arithmetic intensity, and roofline analysis and complete: Profile an attention kernel. Part of the “Turn Training into a S
Learning objectives
- Explain what problem “GPUs, kernels, and the memory hierarchy” solves without hiding behind terminology.
- Trace the variables and causal links across HBM, SRAM, arithmetic intensity.
- Complete “Profile an attention kernel” and judge the result with evidence rather than intuition.
Core concepts
HBM
HBM is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
SRAM
SRAM is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
arithmetic intensity
arithmetic intensity is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and roofline analysis
and roofline analysis is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Profile an attention kernel
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.