Learn AI by Building
From your first dataset to production agents — deep-dive series, hands-on notebooks, and experiments you can rerun yourself.
Tutorials
View All →LLM Agent Cookbook
Build AI agents from scratch — ReAct, Tool Use, Multi-Agent orchestration
ML Cookbook
Master machine learning algorithms with hands-on Jupyter projects
Data Analysis Cookbook
SQL, Pandas, Statistics — everything for data-driven decisions
Ontology & KG Cookbook
RDF, OWL, Neo4j, and GraphRAG for knowledge-powered AI
Paper of the Week
All Issues →Paper of the Week #3 — Half the FLOPs Is Not Half the Time
One integer halves a fine-grained MoE's expert compute (arXiv 2609.04575) and its Table 5 replicates on one A100 to within a point. The paper never reports time, so I measured it: nothing in HF transformers, nothing at batch 1 in the vLLM you run today, 1.35x at batch 8. Plus the OLMoE control and the iso-cost harness control promised in issue #2.
Premium Series
Our Products
Courses and starter kits built from what we measure here
Video Courses
12 hands-on courses — quantization, diffusion, RAG, on-device AI. $199 lifetime bundle, first 3 lectures of every course free
LLM Quantization and Compression
GPTQ, AWQ, GGUF, QLoRA — fit LLMs into the memory you have. The course behind our KV-cache measurements
Starter Kits
Solution notebooks for the free cookbooks — LLM Agent, Data Analyst, ML, Ontology & KG. One-time purchase
Premium Series
140 deep-dive posts across 21 series, bilingual KO/EN, with production-ready code and notebooks
Starter Kits
View All →Practice notebooks, interview questions, and project solutions — ready to download.
Browse Starter KitsLatest Posts
View All →
Same Model, Five Ways to Build the Input: Only Retrieval Moved It Significantly
On BANKING77 with GPT-5.6 Terra held fixed, reordering labels, grouping them and changing reasoning effort each moved one or two items in both directions against the baseline, with no significant paired difference. Attaching five retrieved training examples scored 146 of 154, fixing 19 items and breaking none (exact McNemar p about 3.8e-6).
llama.cpp -ctk and -ctv: The Default CUDA Build Supports Four Values
llama-bench accepts eight KV cache types and llama-server accepts nine, but the default CUDA build compiles FlashAttention kernels for only f16, bf16, q8_0 and q4_0, and only when K and V match. Every other setting has no kernel and prefills at 83-284 tokens per second against about 4,600. Measured on llama.cpp 69320fe on an A100.
Jev's Speed Claims: Benchmarking Label-Only Alternatives on BANKING77
Four ways to get only a label on BANKING77's 77-intent test set, compared on the same 154 messages. Pretrained MiniLM embeddings with a logistic regression reached 90.3% accuracy at a 6.8 ms median on CPU. Jev itself has not been measured yet.
llama.cpp KV Cache Quantization: Why q8_0 Costs 9% of Throughput — or 22%
Mainline llama.cpp on one A100, Qwen3-8B Q4_K_M, llama-server with 1 to 32 concurrent slots. On a 32K prompt, q8_0 cost 9% of server throughput when each request generated 128 tokens and 22% when it generated 1,024, because prefill dominates the short case and prefill is unaffected by the KV type. Per-token decode was 34% slower, matching llama-bench. VRAM in use after startup fell from 41.0 GiB to 25.1 GiB at four 64K slots.
llama.cpp KV Cache Quantization, Measured on One A100 — q8_0 Is Free at 4K and Costs Half Your Decode Speed at 64K
Mainline llama.cpp, Qwen3-8B, one A100: -ctk q8_0 -ctv q8_0 matches f16 perplexity and cuts the 32K cache by 2.1 GiB, but decode at 64K depth drops to 55% of f16 (q4_0: 50%). Two other settings, q5_1 and a q8_0/q4_0 mix, silently ran prefill on the CPU at 43 and 63 tokens per second.
KV Cache Reduction, Measured on One A100 — Part 1: The Twelve Techniques Don't Pay in the Same Currency
Every list of KV cache techniques presents a dozen of them as peers. On one A100 the spread is enormous: prefix reuse cut a shared-prefix workload from 59.4s to 6.2s, while tuning the paged block size moved capacity by under 1%. The reason you cannot rank them on one number is that they pay out in different units.