Learn AI by Building
From your first dataset to production agents — deep-dive series, hands-on notebooks, and experiments you can rerun yourself.
Paper of the Week
All Issues →Paper of the Week #8 — Agents Beat All 20 Science Benchmarks, and None of 135 Follow-up Papers Passed
This week's roundup. Agent4Sci reports that frontier agents beat human-verified baselines on all 20 scientific ML benchmarks it collected, mostly through engineering; Into the Unknown reports that none of 135 agent-written follow-up papers met every condition set by the scientists who posed the problems. Also: BizAgentBench, which grades goals and procedure separately on payment-company workflows, and CarryOn, which rebuilds an agent's context from what it has produced instead of its chat history.
Premium Series
Our Products
Courses and code built from what we measure here
Video Courses
13 hands-on courses — quantization, diffusion, RAG, on-device AI, decision models. $199 lifetime bundle, first 3 lectures of every course free
LLM Quantization and Compression Hands-On
GPTQ, AWQ, GGUF, QLoRA — fit LLMs into the memory you have. The course behind our KV-cache measurements
Free Benchmark Code
The scripts and raw results behind the measurement posts, starting with the KV cache harness. Free with an account
Premium Series
140 deep-dive posts across 21 series, bilingual KO/EN, with production-ready code and notebooks
Latest Posts
View All →
Paper of the Week #8 — Agents Beat All 20 Science Benchmarks, and None of 135 Follow-up Papers Passed
This week's roundup. Agent4Sci reports that frontier agents beat human-verified baselines on all 20 scientific ML benchmarks it collected, mostly through engineering; Into the Unknown reports that none of 135 agent-written follow-up papers met every condition set by the scientists who posed the problems. Also: BizAgentBench, which grades goals and procedure separately on payment-company workflows, and CarryOn, which rebuilds an agent's context from what it has produced instead of its chat history.
Harness² Selector Code: What It Sees and What It Never Sees
Reading google-research/harness2 (commit 1cfe254): the step that picks one of k harness candidates with no answer key sees only the request, the source material and anonymized, untruncated outputs. A failed pick records the first final by label but is left out of scoring, and when a successful pick leads the reflector to write a lesson, the pick's stated reason is stored verbatim in the library next to the adopted harness.
Harness²: A Loop That Edits the Harness Without an Answer Key
Harness² (arXiv:2610.14224) makes the harness around a model eight editable parts and recurses on it with no ground truth: 11.5 points on WorkBuddy-Bench from one improvement step (GLM-5 under OpenCode), tested on the OpenCode and Codex substrates, and a selection step, active only above k=1, that the authors admit can pick the weaker candidate. We read the paper and ran the released demo locally.
Claude Code, Codex, OpenCode, Hermes: How Many of the Eight Harness Parts Do They Open Up?
We compared four agent harnesses (Claude Code, Codex, OpenCode, Hermes Agent) against the eight parts named in the Harness² paper, checking in each tool's official documentation whether a part is exposed as a file or setting, and opening some of the installed tools ourselves. In the docs we read, none of the four opens all eight.
What Is an Agent Harness, and What Does Harness Engineering Do?
An agent is a model plus a harness: the prompts, rules, memory, tools and scripts around it. Where the term harness engineering comes from, the eight parts Google's Harness² paper edits, and what the practice looks like in a repository you already have.
DeepSeek's Config Says 16 KV Heads, Its Cache Holds a Ninth of That: Seven Ways LLMs Handle Attention, and Which Models Use Each
DeepSeek-V2-Lite lists 16 KV heads, which predicts 8,640 MiB of cache at 32K; llama.cpp allocated 972 MiB. A guide to MHA, MQA, GQA, sliding window, MLA, sparse attention and linear-attention hybrids: what each stores, which models use it (checked against their config.json), and what it gives up.