Show a Local Model Buggy Code and Its Tests Protect the Bug, Even When It Notices

Show a Local Model Buggy Code and Its Tests Protect the Bug, Even When It Notices

A common way to use a coding model is to paste a function and ask for unit tests. The tests run, they pass, and coverage goes up. But the model only sees the code you gave it. If that code has a bug, what does the model treat as correct?

A paper accepted at ISSTA 2026, Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests (Zhao, Zhou and Cohen, University of Toronto), measured this on Java. It names two kinds of tests:

• A misguided test passes on the buggy version and fails on the fixed version. It asserts the bug as if it were the right answer.
• An effective test fails on the buggy version and passes on the fixed version. It catches the bug.

On 233 real bugs from Defects4J, averaged over 13 model setups (11 models, some with reasoning on and off), the paper reports that showing the buggy code instead of the fixed code raised misguided tests from 0.46% to 3.84% of all generated tests and cut effective tests from 8.51% to 2.98%. It also proposes a fix: remove the code from the test prompt and give the model only a specification docstring, written by the same model from the buggy code.

The models in the paper are large hosted ones, and the code is Java. We wanted to know whether the same thing happens with a small model people run on their own machine, writing Python tests.

How we measured

• Problems: all 164 Python problems in HumanEvalPack (revision 9a41762). Each has a human-planted bug, the correct version and the original docstring. We did not make the bugs. We first ran the dataset's own reference tests: every buggy version fails them and every fixed version passes, so no problem was excluded.
• Model: Qwen3.5-9B Q4_K_M in llama.cpp (llama-server, commit 4da6337), temperature 0, thinking off, up to 4,096 output tokens.
• Prompts: the paper's test-generation request translated from Java/JUnit to Python/pytest ("Please write some unit tests ... with maximizing both branch and line coverage ..."), plus one line telling the model how to import the function.
• Conditions, one generation per problem each:

- Buggy code: the function with the buggy body, no docstring.

- Fixed code: the function with the correct body, no docstring.

- Model-written spec: no code. Instead, a docstring the same model wrote after seeing the buggy code, using the paper's prompt, which warns that "the function code ... might be buggy".

- Three more for reference: signature only, the original human docstring only, and the buggy code with the original docstring attached.

- We removed the docstring from the main conditions because HumanEval docstrings are themselves a correct specification. With one present, the model would not be "writing tests from the code".

• Scoring: each generated test function (each parametrized case counts separately) runs against the buggy and the fixed version in separate processes, with a 10-second limit per test. Every test lands in one of four boxes: misguided, effective, passes both, fails both. Rates are over all tests in a condition, as in the paper.
• Rules, fixed before running: four main comparisons, each with a problem-level paired bootstrap (10,000 resamples). With four comparisons we use 98.75% intervals, and a difference counts only if its interval excludes zero.

Two scorer fixes went in after a three-problem smoke test and before the main run, both recorded in the plan. The model followed the "output format is Markdown" instruction by labeling its code blocks markdown, which our first extractor missed. And a test file that fails to import now counts as zero tests instead of one failed test.

The bug leaks into the tests

Both intervals exclude zero. With the buggy code in front of it, the model wrote more tests that assert the bug and fewer that catch it. That is the paper's direction, on a 9B model running locally and on Python. Our rates are higher than the paper's on both sides. With a different language, benchmark and model we only compare directions, not levels.

The same problem counted by function rather than by test: from the fixed code, at least one test caught the bug in 141 of 164 problems. From the buggy code, 81.

Here is what that looks like in one problem. get_positive should return the positive numbers in a list. The buggy version returns the negative ones. The model saw only the signature and the buggy body, and wrote seven tests:

assert get_positive([1, 2, 3]) == []
assert get_positive([-1, -2, -3]) == [-1, -2, -3]
assert get_positive([1, -2, 3, -4, 5]) == [-2, -4]
assert get_positive([]) == []
assert get_positive([5]) == []
assert get_positive([-5]) == [-5]
assert get_positive([0, 1, -1]) == [-1]

All seven pass on the buggy code. Fix the bug (e < 0 to e > 0) and six of them fail. A function called get_positive, given only positive numbers, is required to return nothing, because that is what the code did. The model trusted the buggy code over the function's own name. Shown the fixed code instead, the same model wrote assert get_positive([1, 2, 3]) == [1, 2, 3].

In a real project these tests would go green, get merged, and then fail the day someone fixes the bug.

The paper's fix did not help here

Both intervals include zero, so with 164 problems we can't say the spec changed either rate. The point estimate for misguided tests even went up. The paper found the opposite direction on its models: on average, misguided tests fell from 3.84% to 2.69% and effective tests rose from 2.98% to 4.50%. We could not confirm that on Qwen3.5-9B.

To see why, we read the specs. This was not in the plan: we added it after the main results, recorded the change and its labeling rules in the plan's change log before labeling, and treat it as an explanation, not a verdict. Following the paper's section 4.4, each of the 164 specs got two labels: does it describe the bug as intended behavior, and does it describe the correct behavior for the part the bug changes. A language model (Claude) applied the labels with the bug diff in front of it, and we checked 20 random labels by hand. We agreed with 17. In the other three (problems 72, 129 and 149) the spec describes a different function altogether, and we would have labeled them "neither" rather than "kept the bug".

With our own judgment on the three disputed specs, the counts would be 80, 70 and 14, so "about half" still holds.

Half the specs wrote the bug down as the intended behavior, despite the prompt's warning. For get_positive the spec opens with "Extracts all negative integers from the provided list." Once a spec says that, a test written from the spec asserts it. When the spec recovered the correct behavior, misguided tests almost disappeared.

The split is not a clean cause, though. On the problems whose spec kept the bug, tests written from the buggy code itself were also misguided more often (27.3% against 6.2% on the problems whose spec recovered the fix). Some bugs are simply easier to mistake for intent, and those stay mistaken whether the model sees them as code or as its own summary.

The paper made the same observation on bigger models. With its more detailed "advanced" spec prompt (we used the base prompt), the authors found that 48 of Gemini 2.5 Pro's 318 specs (15.1%, our calculation from their counts) and 66 of Qwen3-Coder-Plus's 318 (20.8%) kept the bug. They conclude that removing the code is not enough and that recovering the correct behavior is what matters. Our 9B model, with the base prompt, kept the bug in about half its specs, and the fix had nothing to work with.

SOTAAZ course

This course alone $79 · 24 lectures · first 3 freeWatch free →

What did help, without a verdict

Three more conditions were in the plan only to report, not to judge. Read them as context:

The last row is the realistic one: buggy code that still carries its original, correct docstring. In that condition misguided tests were 4.8% instead of 16.4%, and a test caught the bug in 147 problems instead of 81. Here a human-written description of what the function should do outweighed the buggy body. A description the model wrote by reading the buggy body did not.

What to do with this

If you ask a model to write tests for existing code, the tests check what the code does, and that includes its bugs. A passing suite generated this way says little about whether the code is right. Measured on one 9B model here:

• Give the model the intended behavior in a form that does not come from the code: the docstring, the ticket, an example of expected output. With the human docstring attached, misguided tests fell from 16.4% to 4.8% (a reference condition, reported without a verdict).
• Don't count on the model to summarize its way out of a bug. Its own spec of buggy code carried the bug in about half the problems.
• Read the generated tests whose expected values look odd, like get_positive([1, 2, 3]) == []. Those are where the bug is written down.

With thinking on

Everything so far was with the model's thinking mode off. We reran all six conditions with thinking on and the output limit raised to 16,384 tokens. We had decided in advance to report this as a secondary result, without a verdict. In the paper's table, the seven reasoning setups averaged more misguided tests (4.61%) than the six without reasoning (2.94%). Those averages mostly come from different models, though, so they aren't the same model with thinking switched on and off.

One problem first. With thinking on, many answers didn't finish within the limit. Answers cut off before producing any test numbered 59 of 164 with buggy code, 41 with fixed code and 126 with the signature only. A condition's overall rate then comes only from the problems that finished, and which problems finish differs by condition. So we compared both modes on the same 85 problems: those where buggy code and fixed code produced tests with thinking off and with thinking on. We chose this comparison after seeing the results, and the plan marks it as post-hoc.

Shown buggy code, the thinking model wrote 15.2 points more misguided tests (98.75% interval +7.3 to +22.9, paired bootstrap over problems), and tests that catch the bug almost disappeared, down to 3.3%. Shown fixed code, misguided tests went from 1.9% to 0.0% and tests that catch the bug from 39.6% to 45.0%. That is the paper's direction.

The thinking text suggests why. The get_positive example earlier was with thinking off (six of seven tests misguided). Given the same buggy function with thinking on, the model wrote:

The function is named get_positive but the logic filters for negative numbers (e < 0). This is a bug in the provided code, but my task is to write tests for this specific code, not to fix it. I must test what the code actually does.

It then wrote four tests, including assert get_positive([1, 2, 3]) == [], and three of them were misguided. It noticed the bug and protected it on purpose. The thinking text often traces the code line by line to work out expected values. On problem 13 (greatest common divisor) it ran the loop by hand, wrote that "this implementation is buggy for standard GCD (it returns 0)", and used that 0 as the expected value. The more carefully it follows the code, the more faithfully the bug becomes the expected answer.

We counted roughly how often this happened. Of the 105 buggy-code problems that produced tests with thinking on, a word search finds the thinking mentioning a bug or a mismatch between name and behavior in 28; reading five chosen at random, four (problems 8, 13, 38 and 94) were real recognitions of the bug and one (86) was a generic mention. In 19 of those 28, at least one test was misguided anyway. Their misguided rate was 28.6%, only somewhat below the 40.5% where no bug was mentioned. It's a word-search estimate, so we don't base any verdict on it.

Limits

• One model (Qwen3.5-9B, Q4_K_M), one generation per problem at temperature 0, 164 problems.
• HumanEvalPack bugs are planted by hand into short functions. Defects4J bugs are real bugs in large Java projects. The rates are not comparable, and we compare only directions.
• Up to 18 answers per condition hit the output limit, and 46 generated test files across the six conditions failed to import (counted as zero tests).
• The spec labels come from one model-based labeler with a 20-item hand check, not two independent human annotators as in the paper.
• Thinking off is our main setting. With thinking on, many answers hit the output limit, so we compared the modes only on the 85 problems both finished. Those are problems that finish even with thinking on, so they don't represent all 164.

The plan with its change log, the prompts, the generation and scoring scripts, every generated test with its per-test results, and the spec labels are in the reproduction package below.