vLLM: PagedAttention and continuous batching

DIRECT ANSWER · VERIFIED SOURCES · 2026-07-22

How do PagedAttention and continuous batching improve vLLM serving throughput?

vLLM divides each request’s KV cache into fixed-size logical blocks and maps them through a block table to non-contiguous physical GPU-memory blocks. PagedAttention follows that mapping while computing attention, reducing fragmentation and waste from reserving large contiguous regions. Its scheduler then uses continuous batching to update the batch as requests arrive, finish, or generate tokens, leaving the GPU idle less often.

Lecture alignment

Lecture alignment: separate three concepts—KV cache is the data, PagedAttention is the block-addressing mechanism, and continuous batching is the request scheduler—then draw both a block table and the active sequences at each decode step.

Key takeaways

• Logically contiguous but physically non-contiguous blocks resemble virtual-memory paging and support on-demand allocation and some sharing.
• Less cache waste can fit more concurrent sequences in the same memory, increasing the number of requests available for batching.
• Continuous batching admits and removes sequences dynamically instead of waiting for every request in a static batch to finish.

Boundary and caveat

PagedAttention addresses KV-cache management and access; it is not the same as FlashAttention reducing IO inside an attention kernel. Real throughput gains depend on model, hardware, prompt/output lengths, concurrency, sampling, and latency targets, so paper speedups do not transfer directly to every workload.

Primary sources

• Kwon et al. · PagedAttention and vLLM
• vLLM · Paged Attention Design
• vLLM Project · Source Repository

block tables

block tables is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment before moving to an optimized implementation.

continuous batching

continuous batching is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.

and scheduling

and scheduling is part of the lesson’s causal model. State its inputs, outputs, invariants, and failure mode; then verify it with a hand-check or a minimal experiment.