AI Tools•SOTAAZ Lab••KR

90% of Your Agents Should Just Be Python Scripts: Anatomy of Quadratic Token Costs

We ran the same 10-file daily transaction triage (180 records, 3 anomalies) on Claude Haiku 4.5 two ways. A runtime agent loop burned 102,546 input tokens and 20.7s across 12 turns and failed on turn 12. Asking the same model once (379 input tokens) to write a Python script passed 30/30 runs in 38.8ms each — 7,569x cheaper hourly even with 100% prompt caching.

90% of Your Agents Should Just Be Python Scripts: Anatomy of Quadratic Token Costs

Teams routinely wire a runtime AI agent to a recurring ops job, then get blindsided by the API bill a month later. Even a simple job — scan 10 files, flag bad rows, sum the rest — does not scale linearly inside an agent loop. Because every turn re-sends the growing conversation history, total input tokens explode quadratically ($O(N^2)$) with the number of turns.

We measured how fast that curve actually explodes — and tested the standard counter-argument ("doesn't prompt caching fix this?") — using the exact same model (claude-haiku-4-5-20251001, temperature 0) on the exact same daily transaction triage job:

  • Dataset: 10 deterministic JSON batch files (batch_01.json through batch_10.json, seed 20261121, 180 records total) with 3 injected anomalies (duplicate tx_id TX-0042, negative amount TX-0100, empty customer_id TX-0155), targeting the exact anomaly IDs and the 2-decimal sum of the 177 valid rows (21,848.68).
  • Approach A (Runtime Agent Loop): Give the agent 3 tools (list_files, read_file, write_report) and let it inspect the 10 files turn by turn.
  • Approach B (Code-Gen Once + Plain Python): Show the same model 2 sample rows and 3 rules once, generate a standalone Python script, and execute it 30 times (30 days).

The gap was stark. The runtime agent burned 102,546 input tokens and 20.71 seconds on a single run, then hit max_tokens mid-arithmetic on turn 12 and never called write_report. Meanwhile, the Python script generated once by the exact same model for 379 input tokens (5.15s) passed 30/30 runs in 38.8 ms each with zero rounding drift. Even assuming 100% prompt cache hits on every turn, an hourly runtime loop still costs 7,569× more over 30 days.

Below we walk through the exact 12-turn quadratic ramp and its closed-form scaling, the real prompt-caching math, two common objections (max_tokens and run_python), and a 3-tier decision table with a hybrid fallback pattern.

The 12-turn quadratic ramp, turn by turn

LLM APIs are stateless. Reading batch_01.json on turn 2 means re-paying for batch_01.json on turns 3, 4, 5, all the way through turn 12. All 10 raw files combined are only ~16.5K tokens, yet the 12-turn loop billed 102,546 input tokens — more than 6× the size of the data itself.

Two-panel benchmark chart. Left: per-turn input tokens climbing from 804 on turn 1 to 17,551 on turn 12 (cumulative 102,546 tokens, failing on turn 12) vs 379 input tokens for one-time Python code generation. Right: 30-day Claude Sonnet 4.5 cost on a log scale for daily ($9.85 vs $0.0107) and hourly ($236.44 vs $0.0107) runs.
TurnTool calledInput tokensDeltaCumulative inputLatency
1list_files804—8041.22 s
2read_file (batch_01)948+1441,7521.25 s
3read_file (batch_02)2,610+1,6624,3621.30 s
4read_file (batch_03)4,265+1,6558,6271.26 s
5read_file (batch_04)5,919+1,65414,5461.68 s
6read_file (batch_05)7,597+1,67822,1431.31 s
7read_file (batch_06)9,251+1,65431,3941.27 s
8read_file (batch_07)10,905+1,65442,2991.58 s
9read_file (batch_08)12,580+1,67554,8791.28 s
10read_file (batch_09)14,234+1,65469,1131.25 s
11read_file (batch_10)15,882+1,64884,9951.68 s
12*(truncated summing, no tool call)*17,551+1,669102,5465.62 s

Every file adds roughly 1,655 tokens ($\Delta$) to every subsequent turn. With fixed prompt/tool prefix $S$ and $N$ turns, total input follows the arithmetic series:

$$\text{Total Input} \approx N \cdot S + \Delta \cdot \frac{N(N-1)}{2}$$

Doubling the files quadruples the tokens; 5× the files multiplies input tokens by 22×:

FilesTurns ($N$)Final turn inputTotal input / runSonnet 4.5 cost / run (no cache)
10 (measured)1217,551102,546$0.33
20 (projected)22~35,559~399,993$1.20
50 (projected)52~85,209~2,236,338$6.71

In a 3-file demo, the bill is pennies and nobody notices. In production at 50 files, a single run burns 2.24 million input tokens.

Worse than the bill: failing on turn 12

After burning 102,546 input tokens and 20.71 seconds, the runtime agent still failed. On turn 12, trying to sum 177 floating-point amounts in text exhausted max_tokens=512 before it could call write_report.

Code-gen once + 30 Python runs on the exact same model

MetricApproach A: Runtime Agent (1 run)Approach B: Code-Gen Once + Python (30 runs)Difference
Input tokens102,546 (per run)379 (once ever)270× on run 1
Output tokens1,383 (per run)635 (once ever)—
Execution time20.71 s / run38.8 ms / run (5.15 s gen)534× faster
AccuracyFailed (0 / 1)30 / 30 (100% exact)Deterministic

"Doesn't prompt caching solve this?" — exact math

Applying Claude Sonnet 4.5's official caching rates ($3.75/M cache write, $0.30/M cache read, $15.00/M output) to our exact 12-turn trace (17,551 cache-write tokens, 84,995 cache-read tokens, 1,383 output tokens):

30-day cadence (Sonnet 4.5)Runtime Agent (no cache)Runtime Agent (100% cache hit)Code-Gen Once + CronGap vs cached
1 run$0.3284$0.1121 (-65.9%)$0.010710.5×
Daily (30 runs)$9.85$3.36$0.0107315×
Hourly (720 runs)$236.44$80.68$0.01077,569×

Even with 100% cache hits, an hourly loop still costs $80.68/month for one trivial job — and in production, 5-minute TTLs, dynamic timestamps in system prompts, and context compaction routinely break the cache. Worse, caching never fixes 20-second latency or probabilistic math errors.

Answering the two obvious objections

  1. "What if you raised max_tokens to 4,096?" The agent would likely finish writing out the sum and call write_report, at the cost of another 1,500–2,000 output tokens (~$0.03 extra per Sonnet run) while still risking a single-digit arithmetic slip across 177 floats.
  2. "What if you gave the runtime agent a run_python tool?" That is halfway to Approach B — except you still pay LLM tokens and latency to regenerate the same script 24 times a day, with the risk that one day it writes a slightly broken variant. Once a working script exists, commit it to Git and run it via cron.

The generated script and the hybrid fallback pattern

The script generated in 379 input tokens that passed 30/30 is 25 lines of standard-library Python:

python
import json, sys
from pathlib import Path

data_dir = Path(sys.argv[1])
seen: set[str] = set()
anomalies: set[str] = set()
valid_total = 0.0

for path in sorted(data_dir.glob("batch_*.json")):
    for row in json.loads(path.read_text(encoding="utf-8")):
        tx_id = row.get("tx_id", "")
        amount = float(row.get("amount", 0))
        cust = row.get("customer_id", "")
        is_dup = tx_id in seen
        seen.add(tx_id)
        if is_dup or amount <= 0 or not cust:
            anomalies.add(tx_id)
        else:
            valid_total += amount

print(json.dumps({
    "anomaly_tx_ids": sorted(anomalies),
    "valid_total": round(valid_total, 2),
}))

What if 2–5% of rows genuinely contain messy, unstructured text that a deterministic parser cannot handle? Don't turn the whole job into an agent loop. Process 95% deterministically in Python, collect only the unparseable edge cases, and make one bounded, single-turn LLM call:

python
def triage_with_bounded_fallback(rows: list[dict], llm_classify_once) -> dict:
    clean_total, anomalies, ambiguous = 0.0, [], []
    for row in rows:
        try:
            amt = float(row["amount"])
            if amt <= 0 or not row.get("customer_id"):
                anomalies.append(row["tx_id"])
            else:
                clean_total += amt
        except (KeyError, ValueError, TypeError):
            ambiguous.append(row)

    # Single-turn fallback only when needed — zero O(N^2) loops
    if ambiguous:
        anomalies.extend(llm_classify_once(ambiguous))

    return {"valid_total": round(clean_total, 2), "anomaly_tx_ids": sorted(set(anomalies))}

Where code ends and a runtime agent begins

TierShareTypical workloadsRecommended architecture
1. Structured & recurring~75%Log/JSON/CSV aggregation, deduplication, DB sync, scheduled API jobsGenerate once with a coding agent → commit to Git → run via cron
2. Semi-structured + edge cases~15%Invoice extraction, support triage, alert routingPlain code handles 95% + 1-turn LLM fallback for ambiguous rows
3. True runtime agent~10%Open-ended codebase debugging, ad-hoc research, exploratory triageHard turn cap + tool allowlist + deterministic exit gate

Use LLMs at build time to write, test, and maintain code; run plain code at runtime. Putting a recurring deterministic job inside a runtime agent loop means paying $O(N^2)$ every day to re-derive the same logic — while trading away speed and accuracy.

Stay Updated

Follow us for the latest posts and tutorials

Subscribe to Newsletter