Learn AI by Building

From your first dataset to production agents โ€” deep-dive series, hands-on notebooks, and experiments you can rerun yourself.

Paper of the Week

All Issues โ†’
Issue #10 ยท WeeklyNov 13, 2026

Paper of the Week #10 โ€” Reorganize the Agent's Workspace, and Haiku 4.5 Runs at a Quarter of the Cost

This week: EnvOpt (UC Berkeley, UChicago, Bespoke Labs) lets one agent rebuild the workspace another agent works in, without answer keys. On AutomationBench, Claude Haiku 4.5's cost fell 72.7% and its pass rate rose from 2.33% to 16.67%. The same workspace cut Opus 5's cost by 68.1%, with pass rate moving from 33.83% to 35.17%. We checked every number against the paper's tables and note where the shared figures need context. Also: SlackBench, from the same group, where the best of 33 agent setups scored 80.25% on questions about a research team's Slack, and reading the workspace as plain files beat Slack tools by 7.85 points on average.

Premium Series

Our Products

Courses and code built from what we measure here

Latest Posts

View All โ†’
MiniMax H3 Video + Audio on 8 GB VRAM? Inside FreeVideo's Adaptive Execution Engine

MiniMax H3 Video + Audio on 8 GB VRAM? Inside FreeVideo's Adaptive Execution Engine

We ran FlashML-org/FreeVideo's live Adaptive Execution Planner across 9 hardware configs and aggregated 146 community benchmarks. On 8 GB VRAM + 16 GB RAM, 0 of 50 transformer blocks stay in VRAM and a single 10s 768p clip reads 76.1 GB from SSD plus 173 GB over PCIe; upgrading system RAM to 32 GB cuts SSD streaming to 0 GB.

- AI Tools
Read More
90% of Your Agents Should Just Be Python Scripts: Anatomy of Quadratic Token Costs

90% of Your Agents Should Just Be Python Scripts: Anatomy of Quadratic Token Costs

We ran the same 10-file daily transaction triage (180 records, 3 anomalies) on Claude Haiku 4.5 two ways. A runtime agent loop burned 102,546 input tokens and 20.7s across 12 turns and failed on turn 12. Asking the same model once (379 input tokens) to write a Python script passed 30/30 runs in 38.8ms each โ€” 7,569x cheaper hourly even with 100% prompt caching.

- AI Tools
Read More
Stopping Silent Failures in Production Agents: Don't Let LLMs Grade Their Own Homework

Stopping Silent Failures in Production Agents: Don't Let LLMs Grade Their Own Homework

We tested 20 agent runs (12 silent failures, 8 valid passes) across self-report, two LLM-as-a-Judge setups on Claude Haiku 4.5, and a plain Python code gate. An LLM judge seeing only reports and test logs missed 7 of 12 failures; even with full git diffs it passed a 100+10=120 invoice. The Python gate caught 12/12 in 0.33 ms at zero tokens.

- AI Tools
Read More
Connecting 8 MCP servers burned 47,800 tokens before our first prompt: measurements and a 91% diet

Connecting 8 MCP servers burned 47,800 tokens before our first prompt: measurements and a 91% diet

We measured 8 popular MCP servers (134 tools) with Anthropic's official count_tokens API and Claude Haiku 4.5. Notion alone took 24,441 tokens, all 8 took 47,800. Description compaction saved only 15.6%. An allowlist cut billed input tokens by 81.8% while lifting in-scope accuracy from 10/12 to 12/12; deferred loading cut total input tokens by 91.3% at the cost of a second round-trip.

- AI Tools
Read More
Swapping vLLM for SGLang: the two-card answer held, but on one card 5 of 32 users waited in line

Swapping vLLM for SGLang: the two-card answer held, but on one card 5 of 32 users waited in line

Same Qwen3.8-27B and two A100 PCIe cards, swapping vLLM for SGLang 0.5.23. Two-card winners held (split for 1 user, copies for 64), but on one card SGLang capped concurrency at 27, leaving 5 of 32 users in line with 14.4 s TTFT p95.

- Models & Algorithms
Read More
Where Decision Models Read Their Answer: laya, DiffusionGemma, Kev, CLM and a Plain Classifier in Code

Where Decision Models Read Their Answer: laya, DiffusionGemma, Kev, CLM and a Plain Classifier in Code

Five ways to turn a customer message and 77 intent labels into a probability per label, followed through the code of each. On the same 154 BANKING77 messages they scored from 3.2% to 90.3%. laya cuts every label to three tokens, and giving it more room lifts it from 37.0% to 46.1%; the others keep labels whole, place the options differently, and read the score elsewhere.

- Models & Algorithms
Read More