LLM Inference Optimization Part 3 — Sparse Attention in Practice
Sliding Window, Sink Attention, DeepSeek DSA, IndexCache, and Nvidia DMS. From dynamic token selection to Needle-in-a-Haystack evaluation.

In Part 2, we covered KV Cache quantization, compression, and PagedAttention. Those techniques focus on reducing stored data. Part 3 shifts direction and tackles reducing the computation itself with Sparse Attention.
The key question: "Do we really need every token?"
In most cases, the answer is no. In a 128K context, the tokens that the current token actually needs to attend to are only 5–20% of the total.
The Problem with Full Attention

The rest of this series is paid
- ✓ The whole “LLM inference optimization” series: 3 posts in English, 3 in Korean
- ✓ 3 Jupyter notebooks
- ✓ One payment, no expiry. Opens when you sign in with the email you pay with.
The series pass needs no account to pay. The subscription needs a free account.