Learn AI by Building
From your first dataset to production agents — deep-dive series, hands-on notebooks, and experiments you can rerun yourself.
Tutorials
View All →LLM Agent Cookbook
Build AI agents from scratch — ReAct, Tool Use, Multi-Agent orchestration
ML Cookbook
Master machine learning algorithms with hands-on Jupyter projects
Data Analysis Cookbook
SQL, Pandas, Statistics — everything for data-driven decisions
Ontology & KG Cookbook
RDF, OWL, Neo4j, and GraphRAG for knowledge-powered AI
Paper of the Week
All Issues →Paper of the Week #3 — Half the FLOPs Is Not Half the Time
One integer halves a fine-grained MoE's expert compute (arXiv 2609.04575) and its Table 5 replicates on one A100 to within a point. The paper never reports time, so I measured it: nothing in HF transformers, nothing at batch 1 in the vLLM you run today, 1.35x at batch 8. Plus the OLMoE control and the iso-cost harness control promised in issue #2.
Premium Series
Our Products
Courses and starter kits built from what we measure here
Video Courses
12 hands-on courses — quantization, diffusion, RAG, on-device AI. $199 lifetime bundle, first 3 lectures of every course free
LLM Quantization and Compression
GPTQ, AWQ, GGUF, QLoRA — fit LLMs into the memory you have. The course behind our KV-cache measurements
Starter Kits
Solution notebooks for the free cookbooks — LLM Agent, Data Analyst, ML, Ontology & KG. One-time purchase
Premium Series
140 deep-dive posts across 21 series, bilingual KO/EN, with production-ready code and notebooks
Starter Kits
View All →Practice notebooks, interview questions, and project solutions — ready to download.
Browse Starter KitsLatest Posts
View All →
llama.cpp KV Cache Quantization: Why q8_0 Costs 9% of Throughput — or 22%
Mainline llama.cpp on one A100, Qwen3-8B Q4_K_M, llama-server with 1 to 32 concurrent slots. On a 32K prompt, q8_0 cost 9% of server throughput when each request generated 128 tokens and 22% when it generated 1,024, because prefill dominates the short case and prefill is unaffected by the KV type. Per-token decode was 34% slower, matching llama-bench. VRAM in use after startup fell from 41.0 GiB to 25.1 GiB at four 64K slots.
llama.cpp KV Cache Quantization, Measured on One A100 — q8_0 Is Free at 4K and Costs Half Your Decode Speed at 64K
Mainline llama.cpp, Qwen3-8B, one A100: -ctk q8_0 -ctv q8_0 matches f16 perplexity and cuts the 32K cache by 2.1 GiB, but decode at 64K depth drops to 55% of f16 (q4_0: 50%). Two other settings, q5_1 and a q8_0/q4_0 mix, silently ran prefill on the CPU at 43 and 63 tokens per second.
KV Cache Reduction, Measured on One A100 — Part 1: The Twelve Techniques Don't Pay in the Same Currency
Every list of KV cache techniques presents a dozen of them as peers. On one A100 the spread is enormous: prefix reuse cut a shared-prefix workload from 59.4s to 6.2s, while tuning the paged block size moved capacity by under 1%. The reason you cannot rank them on one number is that they pay out in different units.
Paper of the Week #3 — Half the FLOPs Is Not Half the Time
One integer halves a fine-grained MoE's expert compute (arXiv 2609.04575) and its Table 5 replicates on one A100 to within a point. The paper never reports time, so I measured it: nothing in HF transformers, nothing at batch 1 in the vLLM you run today, 1.35x at batch 8. Plus the OLMoE control and the iso-cost harness control promised in issue #2.
The paper stopped at half the experts. A quarter loses four points instead of thirty, and a model trained without renormalization tells you why.
Two cells arXiv:2609.04575 never ran: k₁=2 and k₁=3 on Qwen3.6-35B-A3B, and the whole k₂ trick on OLMoE, which was trained without renormalization. The second one is a clean test of the paper's mechanism.
One integer halves MoE expert compute. We measured the speed the paper didn't: free at batch 8, not at batch 1.
We reproduced Table 5 of arXiv:2609.04575 on one A100 and measured the throughput the paper leaves out: nothing in HF transformers, nothing at batch 1 in stock vLLM, 1.35× at batch 8.