LESSON 6.3 · Theory · 120 min
Quantization: numbers, error, calibration
Quantization: numbers, error, calibration. Learn INT8/INT4, weight-only methods, GPTQ, AWQ, and GGUF and complete: Compare quality and speed across four quantization methods. Part
Learning objectives
- Explain what problem “Quantization: numbers, error, calibration” solves without hiding behind terminology.
- Trace the variables and causal links across INT8/INT4, weight-only methods, GPTQ.
- Complete “Compare quality and speed across four quantization methods” and judge the result with evidence rather than intuition.
Core concepts
INT8/INT4
INT8/INT4 is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
weight-only methods
weight-only methods is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
GPTQ
GPTQ is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
AWQ
AWQ is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and GGUF
and GGUF is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Compare quality and speed across four quantization methods
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.