Researchers Find That Code-Generating Models Recommend Software Packages That Do Not Exist

Researchers at the University of Texas at San Antonio, the University of Oklahoma and Virginia Tech published a paper in the proceedings of the USENIX Security Symposium of August 2025 on what they call package hallucination. It "occurs when an LLM generates code that recommends or contains a reference to a package that does not actually exist." The study looks at code in two languages, Python and JavaScript. Parts of the paper follow, not the whole.

How an Invented Name Could Become an Attack

An adversary, the paper says, can exploit package hallucinations, "especially if they are repeated, by publishing a package to an open-source repository with the same name as the hallucinated or fictitious package and containing some malicious code/functionality." Users who are later recommended the same name "end up downloading the adversary-created malicious package, resulting in a successful compromise." The paper calls this "a variation of the classical package confusion attack".

"Trivial cross-referencing methods (i.e., comparing a generated package name with a list of known packages) are ineffective for detecting a package hallucination attack," the paper says, "as an adversary may already have published the hallucinated package with malicious code."

That is the quiet part. Once someone has registered the invented name, a check that asks only whether a package is real passes it.

How Often It Happened

In their first experiment the researchers ran 30 tests, with 16 models for Python and 14 for JavaScript, and produced 576,000 code samples. Those tests "generated a total of 2.23 million packages in response to our prompts," and 19.7% of them were determined to be hallucinations, "including 205,474 unique non-existent packages."

The abstract gives the average percentage of hallucinated packages as at least 5.2% for commercial models and 21.7% for open-source models. Python code, the paper says, "resulted in fewer hallucinations than JavaScript."

How Often a Name Came Back

To find out whether hallucinations are "random error or repeatable phenomena," the researchers took 500 prompts that had produced a hallucination and asked each one again 10 times. In that test, 43% of the hallucinated packages were repeated in all 10 queries, while 39% did not repeat at all.

And 58% of the time, the paper says, a hallucinated package was repeated more than once in 10 iterations, "which shows that a majority of hallucinations are not simply random errors, but a repeatable phenomenon that persists across multiple iterations." A persistent hallucination, it adds, "is more valuable for malicious actors looking to exploit this vulnerability".

For anyone who reviews such code, the repetition is the point. A name that is invented again and again is a name an attacker can expect to be asked for.

Mitigations, Limits and One Step Not Taken

The researchers tried several ways to reduce the problem, and "all the mitigation strategies we implemented resulted in a reduced rate of package hallucination". One came at a price: "fine-tuning comes at the cost of diminished code quality."

"Since the writing of this paper, several more advanced models have emerged for code generation," the researchers write, and these "may offer improved performance and different hallucination tendencies and characteristics, which we may have been unable to capture in our study." The study also "includes fewer commercial models due to funding constraints, meaning that the findings may not fully represent the performance and hallucination tendencies of the latest commercial LLMs."

One course of action they chose not to pursue "for ethical reasons" was "publishing actual packages using hallucinated package names to PyPI (with no actual code)."