Researchers Say Code Agents' Patches Can Pass All Test Cases but Contain Vulnerable Code

Code agents are AI systems that, in one paper's words, "autonomously read, generate, test, and submit code." That paper, in the Proceedings of ACL 2026 of July 2026, asks whether a fix from such an agent can be trusted because it passes its tests.

The authors argue that passing all test cases is not a sufficient test. They name a problem: the "functionally correct yet vulnerable" patch. Such patches "pass all test cases but contain vulnerable code." Parts of the paper follow, not the whole.

Correct Is Not the Same as Safe

The paper's motivating study used ordinary conditions, with no attacker. The authors ran one agent pipeline, with four models behind it in turn. They worked on SWE-bench, a benchmark built from real software repositories. They kept only the patches that resolved their issue and passed the full repository test suite. Then they screened those patches for security issues.

Some still had them. The share of vulnerable fixes among those correct patches ranged from 4.3% for one model to 6.0% for another.

In its evaluation, the paper judges vulnerability with an AI model used as a judge. The judge is given, among other things, the official definition of the weakness type and the patch, and returns a verdict. The paper calls this an "initial vulnerability screening."

The weakness types come from the CWE database, short for Common Weakness Enumeration. Among the paper's conceptual examples, a login fix works but logs each user's password. A profile page shows the right text, but leaves its HTML unescaped, so a script could be injected.

An Instruction in the Issue

The authors then asked whether an attacker could make this happen on purpose. Their attack adds an instruction to the description of an issue. It presents the weakness as harmless developer guidance, tied to a plausible aim such as richer logging. In the paper's threat model, the attacker reaches the agent only through public interfaces, such as posting or editing issue text, and sends a single query.

They tested 12 pairings of model and agent framework. Every pairing was compromised. For one weakness type, information exposure, the attack's success rate was 40.7% on one pairing.

That weakness type showed the largest increase over the baseline, the paper says, because it appears to be a harmless request. "Agents are trained to be helpful and frequently add logging for debugging, making them susceptible to this form of injection." Other weakness types were generally less successful. Generating an eval statement, for example, is usually seen as an unsafe operation, which the agents are trained to avoid.

The paper also says the most capable models showed higher attack success. It suggests stronger instruction following can make them more open to following a malicious instruction too.

What Did Not Stop It

The authors tried a simple defense. They added a safety instruction to the agents' system prompt, asking them to avoid bugs or risky patterns while keeping things secure and private. It slightly reduced the attack's success. But the paper says prompt-level defenses "reduce but fail to restore original security."

They also tested a variant in which the injected instruction was present, but the agent's steps were replayed from a clean run. The agent found the right files and reasoned correctly. The weaknesses still showed up in the final patches. The authors attribute this to contamination of the model's internal state. They add a limit: that conclusion is mostly based on the agent's external behavior, since they did not trace the model's internal representations.

From these results, the paper argues that defenses focused only on monitoring an agent's observable behavior are insufficient. Its limitations section also says its main experiments covered four weakness types and some kinds of injected text, and that it is unclear whether the results hold for others. Real development, it adds, often involves people working with the agent.

If a team's rule for merging is a green test run, which check in that pipeline is looking for what a patch should not do?