Learn AI by Building
From your first dataset to production agents — deep-dive series, hands-on notebooks, and experiments you can rerun yourself.
Paper of the Week
All Issues →Paper of the Week #8 — Agents Beat All 20 Science Benchmarks, and None of 135 Follow-up Papers Passed
This week's roundup. Agent4Sci reports that frontier agents beat human-verified baselines on all 20 scientific ML benchmarks it collected, mostly through engineering; Into the Unknown reports that none of 135 agent-written follow-up papers met every condition set by the scientists who posed the problems. Also: BizAgentBench, which grades goals and procedure separately on payment-company workflows, and CarryOn, which rebuilds an agent's context from what it has produced instead of its chat history.
Premium Series
Our Products
Courses and code built from what we measure here
Video Courses
13 hands-on courses — quantization, diffusion, RAG, on-device AI, decision models. $199 lifetime bundle, first 3 lectures of every course free
LLM Quantization and Compression Hands-On
GPTQ, AWQ, GGUF, QLoRA — fit LLMs into the memory you have. The course behind our KV-cache measurements
Free Benchmark Code
The scripts and raw results behind the measurement posts, starting with the KV cache harness. Free with an account
Premium Series
140 deep-dive posts across 21 series, bilingual KO/EN, with production-ready code and notebooks
Latest Posts
View All →
Why LLMs Forget, and How Google's 'Sleep' Moves Memories Into a New Slot
Teach a model a second set of facts and it loses the first. Google's Sleep paper copies the fast memory into a new small slot before it is overwritten, then practices on examples it writes itself. In our 100-fact toy, a sleep step kept 73.2% of the old facts against 50.1% without it, better in all 20 runs, although our first attempt made forgetting worse. The paper reports AIME 2024 at 79.2 vs 73.8 for the base Qwen3-8B and has not released code.
Attention Sink: Why Qwen3-8B Gives Up to 77% of Its Attention to the First Word, and Why 'Banana' Works Just as Well
We measured where Qwen3-8B looks: from layer 7 on, 26-77% of attention goes to the first token, and swapping that token for ' banana' changes almost nothing (0.4588 vs 0.4577). Five papers explain why: softmax must spend attention somewhere, the first token is the one spot every token can see, the spot works as a no-op, streaming breaks without it, and a gate on the attention output cuts it from 46.7% to 4.8% in the model the paper analyzes. Notebook included.
Local AI Coding Agent, Part 3: When the Context Runs Out, the Agent Says It's Done
We ran a local coding agent six times with a 4,096-token context. Four runs ended with pytest passing on a repo nobody had changed: the model's answer was cut off at the end of the window, and agent.py read an answer with no tool call as 'done'. Adding two things, clearing old tool output and dropping cut-off answers, took the agent to the end in 3 of 3 runs at 8K with all 6 hidden tests passing (the Part 2 code stopped at the server in all 3). The price: 24 of the 26 requests right after a clearing reprocessed more than half their prompt, and at 4K two of three runs ran out of things to clear while the third hit the step limit.
Local Coding Agent Part 2: Which Commands Run Without Asking, and Tests the Agent Cannot Rewrite
Three rules for a local coding agent's harness, each tested on a todo app. A first allowlist let 0 of 23 shell commands through, because the model chains them; dropping redirects, checking each piece and adding four words to the list let 10 of 16 run without asking, and every command that asked had rm, cd or an outside path. The existing tests became read-only (put back once), and pytest runs when the model stops. Code, six runs, and the limits.
How Big a Model Can You Fine-Tune on an 8, 12, 16 or 24 GB GPU? We Ran Every Answer Under the Card's Limit
Qwen3 0.6B to 14B, fine-tuned with LoRA, LoRA-FA and VeRA at batch 1 with gradient checkpointing, every answer checked under the card's memory limit. With a bf16 base at 1,024 tokens, an 8 GB card trains 1.7B and a 24 GB card 8B; with a 4-bit base, 4B and 14B. The variant never changed the answer: the largest gap between them was 0.023 GiB allocated, 0.043 GiB reserved.
How to Build a Local AI Coding Agent from Scratch, Decision Model Included (Zero Cloud Calls)
Build a coding agent from three local parts: Flint on llama-server writes the code, a 135-line standard-library Python harness runs its tool calls, and Jeff, a 0.8B decision model on the CPU, notes every shell command before you answer y/N. Five steps, the full code, one real run on a todo app (6 of 6 hidden tests passed), and what it still got wrong.