Paper of the Week #5 — Stale Caches, Burst FLOPS and Decision Models Going Local
This week: Context Language Models reuse a stale KV cache after editing their own context, two ways to run Jev-style decision models yourself, sustained GPU FLOPS 8 to 17 points below the burst number on four NVIDIA cards, and a map eval where always answering water scores 66%.

Sep 29 – Oct 3, 2026 · Context Language Models · Cloudflare Clef · llama.cpp decision models · MSMF · land-or-water eval · SWE-sweep · EasyPPO · Sampling to Reinforce
Seven things people argued about this week. One thread runs through most of them: the headline number and the number underneath it are not the same number. A burst benchmark against a sustained one, a relative gain against the absolute points, a vendor's table against what you can run yourself, an eval score against what you get by always answering "water". For each item: what came out, the numbers we checked in the source, what to watch for, and whether you can check it yourself.
Context Language Models: reuse the stale cache
What came out. Context Language Models (Rulin Shao and 12 co-authors at the University of Washington, Meta Superintelligence Labs, MIT and Trillium Labs; arXiv 2609.37725, code under CC BY-NC 4.0, so not for commercial use) let the model edit its own context as it works: drop a stale block, replace it with a summary, keep going. The abstract reports 11.4% higher accuracy on BrowseComp-Plus with 21.5% fewer FLOPs, and a 47.6% improvement for Qwen3.5-9B on the same benchmark after online RL. Both are relative. In points, the first is 59.4% against about 53.3% for the strongest baseline, and the second is 28.8% to 42.5%.
The part worth reading. Not the headline but Suffix Cache Reuse. After an edit, the tokens that come after it would normally be re-encoded, because their keys and values were computed against a context that no longer exists. The paper keeps those stale states as they are and reports matching standard SGLang at 65.0% of its prefix-reuse FLOPs, adding that stale cache states "can even be beneficial in some cases". The main result is on one benchmark and one model (BrowseComp-Plus, Qwen3.6-27B); an appendix extends it to ordinary chat. That extension matters, because a footnote points out that ordinary chat already does this kind of edit: Qwen3.6-27B's default chat template and GPT-5.6 Sol through the stateless Chat Completions API drop earlier turns' reasoning tokens, so everything after them is re-prefilled. Suffix Cache Reuse is not specific to these models.
Watch for. "Compaction is solved" is not what the paper shows; the stale-cache result rests on one benchmark and one model. The 35% saving the abstract reports is server-side compute at matched performance, not end-to-end speed.
Check it yourself. Yes, on one GPU: take a small Qwen, edit the middle of a long document, and compare full re-prefill, prefix-only reuse and stale-suffix reuse on answer accuracy and first-token KL. This is the item we want to measure properly; it goes into the KV cache series as its own post.
Decision models go local: Cloudflare Clef and llama.cpp
Two releases in two days moved Jev-style decision models, which return a probability per option instead of text, onto hardware you own.
Clef. Cloudflare released Clef (Qwen3.8-27B kept frozen, with a routing head and rank-256 low-rank adapters trained on top) and Clef-flash (the same on Qwen3.5-9B) as open weights under Apache-2.0 (blog). Cloudflare's own table puts Clef ahead of Jev on BANKING77, 94.20 against 79.74, and behind it on When2Call (72.37 against 80.97) and BRIGHT (45.91 against 47.52). Clef-flash scores 66.77 on CLINC150 with out-of-scope questions, against 97.43 for Clef and 89.27 for Jev. Training pairs cross-entropy with a Brier loss, but we found no table of how well the output probabilities are calibrated; the model card's ForecastBench Brier row is a benchmark score, not that. On Hacker News the title was corrected from "open source" to "open weights" (the blog still says open-source), and a commenter put hosted Clef at $0.24 per million input tokens against Jev's $0.042.
llama.cpp. llama.cpp merged decision-model support on 2 October (HF blog, PR #29818), with the same /v1/systemone request shape as Jev. The blog lists Julia-1 (144M, 3 ms), Laya (421M, 5 ms), Kev-4B (12 ms), Lev (4B, 36 ms) and OpenJev 27B (43 ms, CC BY-NC 4.0) on one RTX PRO 6000, and says Clef is next. The PR adds the decision head to the graph and compares llama.cpp's probabilities with each reference implementation; the largest difference ranges from 0.0002 (OpenJev) to 0.022 (Julia-1). That is a check on one input that the conversion is faithful, not a check of accuracy or calibration.
Watch for. Cloudflare's latency (Workers AI: Clef 209 ms, Clef-flash 39 ms, Jev 524 ms), llama.cpp's local latency and Jev's API latency from elsewhere are different measurements; do not put them in one column. For scale on BANKING77: a CPU classifier trained on the labelled data reached 92.9% on the full test set in our runs, and Jev 76.0% on a 154-message sample. Different samples from Cloudflare's, so these place the numbers rather than compare them.
Check it yourself. Yes. We have the same BANKING77 and CLINC150 splits used for Jev, Kev, Jeff and Ollama's models, so Clef-flash and the llama.cpp path can be measured on identical messages. That is the next measurement post.
MAMF and MSMF: a GPU's "achievable" FLOPS was a burst
What came out. Stas Bekman proposed a second number for what a GPU can actually do: MSMF, Maximum Sustainable Matmul FLOPS, the rate once the chip is saturated at its power limit, next to the existing MAMF, which he now describes as the short boost-clock burst. His table (ml-engineering) shows the gap as a share of the theoretical BF16 peak: B300 84.1% to 67.5%, B200 75.7% to 63.5%, H200 84.3% to 76.3%, A100 PCIe 91.0% to 81.7%. The B200 BF16 MAMF run drew 295 W at 1,965 MHz; the sustained run drew 978 W at 1,455 MHz. Short benchmarks finish before the chip heats up.
Watch for. The gap is wider on newer cards (16.6 points on B300, 9.3 on A100 PCIe), the opposite of "newer cards get closer to spec". Each figure is one card, one cooling setup, measured by one person, and the B200 row was measured with its neighbouring GPUs busy while the others ran alone. MSMF is also a single matmul loop; real training mixes in communication and memory waits.
Check it yourself. Yes, in half an hour on one GPU: run a fixed shape briefly, then for twenty minutes while logging power and clocks. The A100 PCIe is in his table and is what we run on, so this becomes an independent re-measurement when the GPU is free.
Land or water: two thirds for answering "water"
What came out. A post asking LLMs whether each point of a latitude-longitude grid is land or water, and plotting the answers as a world map, was widely shared this week. The original idea is an August 2025 blog post, "How Does A Blind Model See The Earth?", that deliberately did not score accuracy. A later repository (egangu/land-water-eval, code under MIT) does, on 16,200 points of a 2° grid against the GSHHG shoreline: Gemma 4 31B base 73.98%, instruction-tuned 82.81% by likelihood and 90.63% with reasoning.
Watch for. Most of a regular grid is ocean. In the repository's own ground-truth file, 10,697 of the 16,200 points are water, so answering "water" every time scores 66.0%. Read 73.98% next to that, not next to 50%. (A grid in degrees also over-samples the poles, so this is not the share of the Earth's surface.) The base-versus-instruction-tuned ranking also differs between sources: the 2025 post called post-training "fairly destructive" for some model variants, while the repository finds the instruction-tuned Gemma ahead.
Check it yourself. Yes, on a laptop with a small base and instruct pair. Compute the trivial baseline on the same mask before anything else.
We later measured it ourselves on Qwen3.5 0.8B to 9B. Bigger models drew the continents better, and asked in Korean with north latitude and west longitude, the 9B's map turned left-right symmetric: it lost east and west. Writing the same coordinates as 37°N 123°W removed the symmetry. One Letter Holds a Language Model’s World Map Together
SWE-sweep: 4.7% without hints, and a question about the scorer
What came out. SWE-sweep (12 authors at Meta, Harvard, the University of Washington and Stanford; swesweep.com, a preprint with no arXiv ID yet, code under MIT) asks an agent to find and fix real bugs in past commits of 100 repositories, with no issue text pointing at them: 4,068 bugs in 22 languages. The best configuration, Sol 5.6 at xhigh effort, solved 4.7% at $72.30 of API cost per repository; Luna 5.6 at xhigh solved 2.5% at $2.24.
Watch for. A fix counts only if the hidden tests from the future commit pass, so an agent that fixes a different, real bug scores zero. The authors remove future git history from the environment, which answers one concern raised this week; the false-negative rate is not analysed. "Agents find 5% of bugs" is not what this measures.
Check it yourself. Only partly; a meaningful score is expensive.
EasyPPO: one run per method
What came out. EasyPPO (Xuanyi Zhou, Qiuyang Mang et al., UC Berkeley and Princeton; arXiv 2609.36802) reports a more stable PPO for LLM post-training, compared with plain PPO, PPO with an actor-only filter, VAPO and HL-Gauss PPO on Qwen3.5-9B.
Watch for. The paper states one run per method and task. The headline gains (14.89%, 2.28% and 9.47%) are relative; in absolute points the best checkpoint is ahead of plain PPO by 1.92, 1.46 and 3.74 on FrontierCS, AIME24 and Search-R1, and ahead of the next-best method by 0.91, 1.46 and 0.89. Stability claims from single runs are the kind that need seeds.
Check it yourself. Not at the paper's scale; a small PPO with three seeds per method takes hours.
From Sampling to Reinforce
Sasha Rush published a tutorial that explains REINFORCE and the KL estimators used in RL post-training by enumerating small discrete distributions exactly, so bias and variance are computed rather than sampled. Worth an evening if you work on RL fine-tuning; it is an explanation, not a new result. By its own account enumeration "cannot scale" to real vocabularies, and the code was written by AI while the text was written by him.
The Ledger
Issue #4 promised two runs for this issue: DeepSeek-V2-Lite as a second non-renormalized control for the MoE result in issue #3, and forcing identical token sequences through every condition. Neither ran. It also promised a main pick with its own measurement, and this issue has none, because Paper of the Week changes shape from this issue on: a weekly round-up of what people argued about, with the numbers checked in the sources, instead of one paper and one experiment. Measurements now go into their own posts, where they have room. The open items move there: the DeepSeek control and token forcing for the MoE work, HoH at ten iterations, and from this week Suffix Cache Reuse, Clef-flash on the same BANKING77 and CLINC150 messages, and an MSMF re-measurement on an A100 PCIe. With no experiment in each issue, the reproduction streak ends here; the measurement posts keep their own record.
