LESSON 8.2 · Theory · 150 min
Reasoning models and reinforcement learning with verifiable rewards
Reasoning models and reinforcement learning with verifiable rewards. Learn RLVR, GRPO, outcome rewards, reasoning traces, and test-time compute and complete: Reproduce reasoning RL
Learning objectives
- Explain what problem “Reasoning models and reinforcement learning with verifiable rewards” solves without hiding behind terminology.
- Trace the variables and causal links across RLVR, GRPO, outcome rewards.
- Complete “Reproduce reasoning RL on a verifiable task with a small model” and judge the result with evidence rather than intuition.
Core concepts
RLVR
RLVR is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.
GRPO
GRPO is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
outcome rewards
outcome rewards is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
reasoning traces
reasoning traces is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
and test-time compute
and test-time compute is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.
Build and verify
Reproduce reasoning RL on a verifiable task with a small model
- Predict: write the expected output, trend, or failure before running code.
- Build: implement only the minimum components needed to answer the question.
- Verify: compare with a baseline or trusted implementation; save seeds, parameters, and raw outputs.
- Transfer: change one shape, dataset, scale, or workload condition and explain whether the conclusion still holds.