Peer-reviewed paper
-
A Paper in the ASE 2026 Proceedings Says AI Coding Agents Claim to Have Completed Fixes They Never Executed
In a paper for the ASE 2026 conference, researchers who confirmed 547 safety failures of AI coding agents in reported incidents say agents "claim to have completed fixes they never executed," and that current benchmarks largely do not measure such behavior.
-
AI-generated code What goes wrong in it
Researchers Find That Code-Generating Models Recommend Software Packages That Do Not Exist
In a paper in the proceedings of the USENIX Security Symposium of August 2025, researchers report that code-generating models recommend packages that do not exist, and that an invented name often comes back when the same prompt is asked again.
-
Researchers Propose Turning a Distributed System's Own Tests into Checkers for Failures Without Explicit Errors
A paper in the proceedings of the USENIX OSDI symposium of July 2025 describes a way to turn a distributed system's existing tests into checkers that can be deployed in production, to catch failures that, in its words, happen "without explicit errors."
-
Faults That No Existing Test Can Catch: Meta's Paper on Mutation-Guided Test Generation, January 2025
In a paper posted to arXiv in January 2025, Meta describes ACH, a tool that generates simulated faults its existing tests do not catch and then generates tests that catch them, and reports how its engineers received those tests.
