Learn AI by Building
From your first dataset to production agents โ deep-dive series, hands-on notebooks, and experiments you can rerun yourself.
Paper of the Week
All Issues โPaper of the Week #10 โ Reorganize the Agent's Workspace, and Haiku 4.5 Runs at a Quarter of the Cost
This week: EnvOpt (UC Berkeley, UChicago, Bespoke Labs) lets one agent rebuild the workspace another agent works in, without answer keys. On AutomationBench, Claude Haiku 4.5's cost fell 72.7% and its pass rate rose from 2.33% to 16.67%. The same workspace cut Opus 5's cost by 68.1%, with pass rate moving from 33.83% to 35.17%. We checked every number against the paper's tables and note where the shared figures need context. Also: SlackBench, from the same group, where the best of 33 agent setups scored 80.25% on questions about a research team's Slack, and reading the workspace as plain files beat Slack tools by 7.85 points on average.
Premium Series
Our Products
Courses and code built from what we measure here
Video Courses
13 hands-on courses โ quantization, diffusion, RAG, on-device AI, decision models. $199 lifetime bundle, first 3 lectures of every course free
LLM Quantization and Compression Hands-On
GPTQ, AWQ, GGUF, QLoRA โ fit LLMs into the memory you have. The course behind our KV-cache measurements
Free Benchmark Code
The scripts and raw results behind the measurement posts, starting with the KV cache harness. Free with an account
Premium Series
140 deep-dive posts across 21 series, bilingual KO/EN, with production-ready code and notebooks
Latest Posts
View All โ
The Transformer Block from 2017 to Mamba Hybrids: 15 Changes, 13 Rebuilt in Code
Fifteen changes to the Transformer block, from Post-LN to Mamba hybrids, each drawn as one diagram and explained with what the paper reported and what it costs. We rebuilt one layer from seven real models as short PyTorch functions; all seven match the Hugging Face output. On the same sentence, the residual stream grows 98-fold from the first layer to the last in Qwen3-0.6B and only 1.1-fold in BERT.

Before voting on truth, let the model calculate: testing Against All Likelihood
Across 18 arithmetic questions and three frames, direct answers scored 0/54; an exploratory calculation-first follow-up scored 54/54. We examine floor and ceiling effects, a scoring repair, and what invariance can establish.

Before mapping thought: 18 of 128 step labels changed when markers disappeared
We collected 16 local reasoning traces, checked annotation stability and compared transitions with shuffled order. Marker removal changed 18/128 comparable labels; this is a measurement audit, not a Mole-Syn training replication.

The harness improved. The arithmetic did not: a fixed-model repair experiment
A local Qwen3.5 9B harness search improved 36/72 to 50/72 correct attempts, mainly by fixing JSON escaping. We inspect frozen selection, unchanged arithmetic failures, and a full-development evidence ablation.

We connected PaperOrchestra to a local model. The PDF compiled; its evidence still needed checking.
A constrained run of official PaperOrchestra components produced a PDF with one changed numeric cell and two unresolved citation keys. A second seed failed at outline parsing. We audit the logs, manuscript and simulated review.

Several OpenMausBot Bots on One Local GPU: Where Each Message Waits
We ran up to eight OpenMausBot bots against one local Qwen server with two slots. Bots wait for a shared desktop that a Local VM bot holds for its whole turn, for the engine to read one prompt at a time, and from the third bot on, every message re-reads about 32K tokens.