AI Hallucinations Create “Slopsquatting” Supply Chain Threat

Developers relying on large language models (LLMs) to build code could unwittingly be exposing themselves to a new type of supply chain attack, security experts have warned.

“Slopsquatting” was first coined by Python Software Foundation (PSF) developer in residence, Seth Larson, according to cybersecurity vendor Socket.

It’s a play on “typosquatting,” a popular tactic used by threat actors for phishing campaigns, where they register slightly misspelled versions of legitimate domains.

In this new take, a threat actor would prompt an LLM to create some code. The code it returns may contain open source software packages that don’t exist – a common problem for AI.

However, the threat actor could then publish a fake package to an official repository with the same details as the hallucinated one and insert malicious code into it. When another user then prompts the same LLM to generate code and it returns the same hallucinated response, the victim would be directed to download the malicious package.

Read more on AI code: Most Cyber Leaders Fear AI-Generated Code Will Increase Security Risks

This is more likely than it sounds, according to a study on package hallucinations from researchers at Virginia Tech and the universities of Oklahoma and Texas.

They tested 16 code-generation LLMs and prompted them to generate 576,000 Python and JavaScript code samples.

The research found that, on average, a fifth of recommended packages didn’t exist – amounting to 205,000 unique hallucinated package names.

More importantly, it revealed that 43% of the same hallucinated packages were suggested every time when re-running the same prompts 10 times each, and 58% were repeated more than once. Just 39% never reappeared.

“This consistency makes slopsquatting more viable than one might expect,” argued Socket.

“Attackers don’t need to scrape massive prompt logs or brute force potential names. They can simply observe LLM behavior, identify commonly hallucinated names, and register them.”

Turning Up the Heat

The hallucinated packages were also “semantically convincing,” making it difficult for developers to spot by sight. Further, they were more likely to be created the higher the “temperature” of the LLM – in other words, if the LLM had been set to create more random responses.

This represents a particular risk for those wedded to the idea of “vibe coding,” where developers are more likely to blindly trust AI content.

“This threat scales. If a single hallucinated package becomes widely recommended by AI tools, and an attacker has registered that name, the potential for widespread compromise is real,” warned Socket.

“And given that many developers trust the output of AI tools without rigorous validation, the window of opportunity is wide open.”

The best way to mitigate slopsquatting is for developers to proactively monitor every dependency and use tools to vet dependencies before adding them to projects, the vendor concluded.