Local Coding Agent Part 2: Which Commands Run Without Asking, and Tests the Agent Cannot Rewrite

Local Coding Agent Part 2: Which Commands Run Without Asking, and Tests the Agent Cannot Rewrite#

The agent from Part 1 asked y/N before every shell command. ls, which lists files, asked the same way as rm, which deletes them. So the first rule we added in this part is this one: commands that only read files or run the tests inside the repo run without asking.

We started simple. We put seven command words on a list (ls, cat, head, tail, grep, wc, pytest) and refused any command that had a symbol such as ;, & or | in it. In this post we call that the first rule. Then we ran the same todo task three times. The model ran 23 bash commands across those three runs, and not one of them ran without asking. All 23 stopped at y/N.

The reason is the way the model writes commands. It writes them like this:

python -m pytest -q 2>&1 | tail -20

This runs the tests (pytest) and shows only the last 20 lines of the output (tail). Both words are on the list. It is two allowed commands joined together, so why did it ask?

Because the first rule looked at the whole line at once. The | joins the two commands, and 2>&1, which asks for error messages too, contains an &. The first rule refused any line with those symbols, whatever surrounded them. Of the 23 commands, 22 chained two or more commands with |, && or ; like this. The 23rd was a bare true, which was not on the list. The rule was safe and useless.

So we changed three things. Redirects such as 2>&1, which only route error messages, are dropped before the check. A chained command is cut at |, &&, || and ;, and each piece is checked against the list. And we added four words the model had actually used (find, echo, pwd, true). That is the second rule. We ran three more times with the second rule, and 10 of the 16 bash commands the model ran went through without asking. Each of the 6 that still asked contained rm (delete a file), cd (change directory) or a path outside the repo. The y/N prompts that were left were the ones worth a person's look.

This post covers the three rules we added to agent.py and what each one did in six runs:

1. Commands that only read or run the tests inside the repo run without asking. Everything else still asks y/N, with the decision model's note, as in Part 1. The first and second rules above are two versions of this rule 1.
2. The tests that were there at the start are read-only, and a new edit_file tool changes one piece of a file instead of rewriting all of it. The guard put a test file back once.
3. When the model says it is done, agent.py runs pytest itself. It passed the first time in all six runs, so the part that hands failures back to the model was never used.

Our six tests that the agent never sees passed in all six runs, as they did in Part 1. These are three runs per rule, every remaining prompt answered y by a script, on one task. We wrote down our predictions and committed them before each set of runs; how they scored is at the end.

Where Part 1 left off#

Part 1 built the agent from three local parts: Flint on llama-server writes the code, agent.py runs its tool calls, and Jeff, a 0.8B decision model on the CPU, writes a one-line note before every shell command. You answer y/N each time.

Running it on the todo app left three problems:

• You approve everything. Every ls asks, the same as every rm. In an earlier run we pressed y nine times, and three of those commands left the repo.
• The model rewrote the test file. In run 3 one added line made its own test file fail.
• It said done without running the tests, in two of three runs.

Each rule below answers one of them. The code is the Part 1 file plus 81 lines, 216 in all, still with only the standard library. Below are the parts that changed; everything else is as in Part 1.

Rule 1: which commands run without asking#

The code below is the second rule, the version agent.py has now.

import re
import shlex

# Commands that only read or run the tests, with no path outside the repo, run without asking. Models chain commands
# (`python -m pytest -q 2>&1 | tail -20`), so a command is split at | && || ; & and every piece must be on the list.
SAFE = {"ls", "cat", "head", "tail", "grep", "wc", "find", "pytest", "echo", "pwd", "true"}


def safe_command(command):
    command = command.replace("2>&1", "").replace("2>/dev/null", "")
    if any(c in command for c in "<>$`\n") or any(f in command for f in ("-exec", "-ok", "-delete", "-fprint", "-fls")):
        return False
    for piece in re.split(r"\|\||&&|[|;&]", command):
        try:
            words = shlex.split(piece)
        except ValueError:  # unbalanced quotes: let a person look at it
            return False
        if words[:2] in (["python", "-m"], ["python3", "-m"]):
            words = words[2:]
        if not words or words[0] not in SAFE or any(w.split("=")[-1].startswith(("/", "~")) or ".." in w for w in words[1:]):
            return False
    return True

and in run_tool, the bash branch now asks only when the command is not safe:

        if name == "bash":
            # Anything that is not a plain read or test run inside the repo still needs a person's yes.
            if not safe_command(args["command"]):
                print(flag_command(root, args["command"]))
                if input(f"run `{args['command']}`? [y/N] ").strip().lower() != "y":
                    return "the user declined to run this command"
            p = subprocess.run(args["command"], shell=True, cwd=root, capture_output=True, text=True, timeout=60,
                               stdin=subprocess.DEVNULL)
            return (p.stdout + p.stderr)[-6000:] + f"\n[exit code {p.returncode}]"

safe_command works in three passes:

• It drops the two redirects models add to almost every command, 2>&1 and 2>/dev/null. Both only decide where error messages go.
• It refuses anything that could write, run something else or reach outside. That means > and <, $ and backticks, which run a command inside a command, a newline, which starts a second command, and the find options that delete files or run commands.
• It splits what is left at |, &&, ||, ; and &, and checks each piece. The first word must be on the list (python -m pytest counts as pytest), and no later word may start with / or ~ or contain ... A path after =, as in --basetemp=/x, is checked the same way.

The six runs used the version just before this one. Reviewing the code after the runs, we found that it let through a command joined by a single & (ls & rm -rf todo.py) and a path after =. The code above fixes both. We fed all 39 commands from the run logs to both versions, and no command got a different verdict.

The decision model's note now appears only on commands that ask. A command that runs without asking does not wait for Jeff.

Compared with the first rule, three things changed. First, the first rule did not drop redirects, so the & and > inside 2>&1 were enough to refuse a command. Second, it did not split anything: any ;, & or | meant a refusal on the spot. Third, its list had seven words; the second rule adds find, echo, pwd and true for eleven. All four are words the model typed in the three runs under the first rule. So the jump from 0 to 10 in the table below comes from the three changes together, not from any one of them.

After the three runs under the first rule (P2-1 to P2-3), we replayed their 23 commands through the second rule: 14 would pass and 9 would ask. We had already read those 23 and written the rule against them, so that replay says only that the rule does what we meant. Before three new runs with the second rule (P2-4 to P2-6), we committed our predictions. The second row of the table is those new runs.

Under the second rule, the 10 that ran without asking were pytest with tail (6), find with head and echo (3), and one echo. The first 6 would pass on the old seven-word list once redirects are dropped and the line is split. The other 4 need the new find and echo. The 6 that still asked:

• cd /tmp && rm -f todos.json && python /app/todo.py add "buy milk" --due 2024-01-15 ..., twice, in P2-4 and P2-5
• pwd && rm -f todos.json && python todo.py add "buy milk" ..., twice
• rm -f todos.json && python -m pytest -q 2>&1 | tail -5 and rm -f todos.json && ls

Every one of them deletes a file, changes directory or names a path outside the repo. P2-6 asked nothing at all: three commands, all pytest or find or echo.

To check your copy, save agent.py and run this in the same folder:

python3 -c "import agent; print(agent.safe_command('python -m pytest -q 2>&1 | tail -20'), agent.safe_command('cd /tmp && rm -f todos.json'))"

If you see this, it worked: True False

What this rule does not do#

It is a check on the words of a command, not a sandbox. cat notes.txt passes, and if notes.txt is a link to a file elsewhere on your disk, cat follows it. Reading the text of a command cannot tell you everything it will touch. Limiting what a running command can reach is the job of the layer below, an isolated environment such as a container, not of a text check. If you need real isolation, run the agent on a copy of the repo inside a container. We ran every run on a copy of the repo, but not in a container.

Running pytest without asking runs code the model wrote. The model writes test files, and pytest imports and runs them. A test file can do anything Python can. So rule 1 does not say "the model cannot change your machine without a yes". It says "the commands it types to read and test do not need a yes". What the tests themselves do, you see only by reading them. In our six runs we read every test file the model left; none touched anything outside the test's temporary folder. Six runs cannot tell you that it never happens.

Rule 2: the existing tests are read-only#

First a new tool, so the model can change part of a file without sending all of it:

    {"type": "function", "function": {
        "name": "edit_file", "description": "Replace one exact piece of text in a file. old must appear exactly once.",
        "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "old": {"type": "string"},
                                                        "new": {"type": "string"}}, "required": ["path", "old", "new"]}}},

        if name == "edit_file":
            path = inside(root, args["path"])
            text = path.read_text()
            if text.count(args["old"]) != 1:
                return f"error: old appears {text.count(args['old'])} times in {args['path']}, it must appear exactly once"
            path.write_text(text.replace(args["old"], args["new"]))
            return f"edited {args['path']}"

Then the guard. At the start of a run, agent.py reads every test_*.py file and keeps the source of each test function. After every tool call, from any tool including bash, it compares again. If a test function that was there at the start has changed or is missing, it writes back the last version of the file that still had them all, and tells the model why.

import ast


def test_functions(text):
    """{name: source} for every test function in a file; None if the file does not parse."""
    try:
        tree = ast.parse(text)
    except SyntaxError:
        return None
    return {f.name: ast.unparse(f) for f in tree.body if isinstance(f, ast.FunctionDef) and f.name.startswith("test")}


def guard_tests(root, kept):
    """Put back any test file whose original test functions were changed or removed. kept holds the last good text."""
    notes = []
    for rel, (original, good) in kept.items():
        path = root / rel
        now = path.read_text() if path.exists() else ""
        found = test_functions(now) or {}
        broken = [name for name, src in original.items() if found.get(name) != src]
        if broken:
            path.write_text(good)
            notes.append(f"{rel} was put back: the existing tests {', '.join(broken)} are read-only. "
                         "Add new test functions instead of changing these.")
        else:
            kept[rel] = (original, now)
    return "\n".join(notes)

The model may still add tests, imports and helpers to the same file. Comparing through ast means a change in spacing or comments does not count; a change in what a test does, does.

It fired once. In P2-1, step 5, the model rewrote test_todo.py whole with write_file and left out test_missing_id, one of the three original tests. agent.py put the file back and added the note to the tool result. In step 6 the model wrote the file whole again, this time with test_missing_id in it, and the run went on.

It also shows what edit_file did not change. The model still rewrote test_todo.py whole in five of the six runs (all but P2-2). In P2-1, P2-4 and P2-5 it used edit_file on the test file too, most in P2-5, seven edits to the test file. In P2-2 it used only edit_file. At the end of all six runs the three original test functions were unchanged.

The guard keeps only the text of the test functions. It cannot stop the model from leaving them alone and making them skip, by adding pytestmark to the file or writing a conftest.py. That did not happen in these six runs.

Rule 3: pytest runs when the model stops#

def pytest(root):
    p = subprocess.run([sys.executable, "-m", "pytest", "-q"], cwd=root, capture_output=True, text=True, timeout=300)
    return p.returncode, (p.stdout + p.stderr)[-4000:]

and in run, the place where the loop used to end:

        if not calls:
            # The model says it is done. Check that ourselves before we believe it.
            code, out = pytest(root)
            print(f"[{step}] done? pytest exit {code}: {out.strip().splitlines()[-1] if out.strip() else ''}")
            if code == 0:
                return (msg.get("content") or "") + "\n" + out.strip().splitlines()[-1]
            messages.append({"role": "user", "content": f"pytest fails, so the task is not done yet:\n{out}"})
            continue

The run function also takes its snapshot of the tests at the start and calls guard_tests after each tool call. Both are in the full file.

The loop still ends when the model stops calling tools. Now it ends only if pytest passes. If it fails, the model gets the failure as a new message and keeps working, within the same 30 steps.

In all six runs pytest passed the first time. So the line that hands back failures never ran. That part is the last two lines of the code above, messages.append(...) and continue.

Something we did not predict: the model ran pytest itself in all six runs. In Part 1, runs 2 and 3 never did. The system prompt and the task text are the same, and the tool list is different (edit_file is new). Six runs cannot tell us whether that is why.

The decision model on what still asks#

Jeff, the decision model, writes its one-line note only on commands that ask y/N. Across the six runs, 29 commands asked. Under the first rule (seven words, refuse any symbol), all 23 bash commands in P2-1 to P2-3 asked; under the second rule (redirects dropped, each piece checked, eleven words), 6 of the 16 in P2-4 to P2-6 did. Within those 29, two kinds of command came out differently:

• Commands with a path outside the repo: 6 of 6 flagged LOOK CLOSELY, from 0.58 to 0.75. In five runs out of six the model left the repo to try the program by hand: cd /tmp && rm -f todos.json && python /repo/todo.py ..., with /repo, /app or the home folder as its guess. None of those todo.py files existed, so each attempt failed with "No such file", but rm -f todos.json in /tmp did run. The script answered y every time. A person reading the yellow line would likely have said no.
• Commands that delete only inside the repo: 3 of 8 flagged. For example rm -f todos.json && ls came back routine (0.32). The question in agent.py asks Jeff to flag commands that "touch paths outside the current repo or delete files", so these should have been flagged. Jeff's number ranged from 0.27 to 0.63 on them.

With the second rule, those are now the only commands you see. Each prompt is more likely to matter, which is the point: in the three runs under the second rule, six prompts, and each one deleted, changed directory or left the repo.

All six runs#

The task, repo, models and settings are the ones from Part 1: the todo app, the six-point feature request, Flint UD-Q2_K_XL with a 32K context on an A100, Jeff 0.8B on 8 CPU cores. Each run starts from a fresh copy of the repo. Every y/N prompt is answered y by a script, as we did by hand in Part 1.

The predictions we committed before each set:

• First rule (P2-1 to P2-3): at most 2 prompts per run, missed (9, 5, 9). The guard puts a test back at least once, hit. The model rewrites the test file whole in at least 2 runs though edit_file exists, hit. At least 2 runs end by themselves with pytest passing, hit (3). Hidden tests 6 of 6 in those, hit.
• Second rule (P2-4 to P2-6): at most 4 prompts per run, hit (3, 3, 0). At least one run proposes a path outside the repo, hit (2). Jeff flags every asked command with an outside path, hit (2 of 2). At least 2 runs end by themselves, hit (3). Hidden tests 6 of 6, hit.

What we did not check#

• Three runs per rule, one task, one model. Temperature is 0.6, so other runs take other steps. These runs show what the rules do, not how often.
• The pytest feedback path. It never fired.
• The rule on other commands. We wrote the allowlist for the commands this model typed on this task. Another model may type commands that are safe and still ask, or the other way round. Read what asks and what does not in your own runs.
• A real 16 GB card. As in Part 1, the 14,386 MiB peak for Flint was measured on an A100.

Next#

Part 3 is about the context window: what to drop from the messages list when a long task fills the 32K, and how to keep the parts that matter. We will write the prediction down before we run it. The newsletter at the bottom of the page will tell you when it is out.