Understanding the Threat
Recent research reveals a critical vulnerability in AI-generated computer code. The study examined 16 large language models, generating 576,000 code samples. It found a staggering 440,000 references to non-existent third-party libraries, termed “hallucinations.” These fictitious dependencies can lead to serious security risks. They create opportunities for supply-chain attacks that can compromise legitimate software, allowing malicious packages to infiltrate systems and steal data or plant backdoors.
Key Findings
- 440,445 out of 2.23 million package references were hallucinated, representing 19.7%.
- Open-source models showed the highest rate, with 21% of dependencies linking to non-existent libraries.
- Dependency confusion attacks exploit these hallucinations, potentially redirecting software to malicious versions of packages.
- 43% of hallucinations were repeated across multiple queries, indicating a pattern that attackers could exploit.
Implications for Software Security
This phenomenon poses a significant risk to the software supply chain. As developers increasingly rely on AI for coding, the potential for hallucinated dependencies to be trusted and installed without verification grows. This can lead to widespread vulnerabilities and attacks on major companies. Understanding and addressing these hallucinations is essential for securing software development and protecting users from malicious threats. Ensuring that AI tools are reliable and trustworthy is crucial as they become integral to modern programming practices.
