Dependency Provenance in AI-Generated Code

Microsoft's CTO has predicted that AI will write 95% of code within five years. Enterprise adoption is already there in part: AI tools are estimated to generate 30 to 40% of code inside large enterprises today. What almost nobody talks about is what those tools do to dependency provenance, meaning the ability to verify where a piece of software actually came from, who built it, and whether that build path can be trusted. Most AI-suggested dependencies fail that test entirely, and closing the gap means treating provenance as a live check built into the coding workflow, not a report generated after the fact.

The core problem sits upstream of anything a vulnerability scanner can catch. A scanner needs a real package with a real identity to check against known CVEs. AI models routinely suggest packages that don't exist, or that exist under a different name, or whose actual maintainer and build path nobody on the team has looked at. Endor Labs' 2025 State of Dependency Management report found that 80% of dependencies proposed by AI coding tools introduce some form of risk, whether through known vulnerabilities, fabricated package names, or provenance that simply can't be traced. That number scales with adoption. Every quarter AI writes more code, the gap between code shipped and provenance verified gets wider, not narrower.

How AI models suggest dependencies and why their training makes provenance invisible to them

Large language models suggest dependencies by pattern-matching against whatever code they were trained on. They are not querying PyPI or npm in real time, not checking a hash, not confirming a package still exists or ever did. The suggestion comes out of statistical association, phrased with the same fluency and confidence whether the package is real or invented.

Version numbers follow the same failure mode. A model can output pandas==2.5.0 or tensorflow==3.2.1 with total confidence, because those strings are plausible in form even when no such release exists. The model has no internal concept of a distribution path: it doesn't know which registry would host the package, under what namespace, built by which system, signed by whom. It's producing a name-shaped token, not a verified artifact.

That distinction matters because of how often the suggestion goes straight into a codebase. Roughly 62% of developers rely on AI tools for dependency selection, which means a model's guess frequently substitutes for a human doing an actual lookup. Provenance, properly understood, requires an unbroken chain from source code to build artifact to distribution point. AI suggestion generates a name with no such chain attached to it at all. Everything downstream, the install, the build, the SBOM, inherits that blank spot.

What the hallucination data actually shows about non-existent package risk

The scale of the hallucination problem is now measured, not anecdotal. Spracklen et al., published at USENIX Security 2025, analyzed 576,000 code samples generated across 16 popular LLMs and found that 19.7% of suggested package dependencies, 440,445 packages in total, were hallucinated: names that corresponded to nothing in any real registry.

Open-source models hallucinated at close to 22%; commercial models came in lower, around 5%. That gap is real and worth noting, but 5% at enterprise scale is still a large absolute count once you multiply it across thousands of developers running thousands of prompts a day. Neither number is something a security team should treat as tolerable, given what a single hallucinated package can turn into once an attacker notices it.

The more unsettling figure in the same study is repeatability. Forty-three percent of hallucinated package names reappeared across multiple, separate prompts, the same fictional name generated reliably rather than as a one-off fluke. That turns a random generation error into a stable, targetable pattern. An attacker doesn't need to guess at what a model might hallucinate; they can run the same class of prompts repeatedly, log the names that keep surfacing, and register them before any developer notices the package doesn't exist.

What the research doesn't yet answer is how often developers actually install these hallucinated names once suggested. The hallucination rate is documented; the downstream adoption rate isn't, at least not at comparable scale. That's a meaningful blind spot, because it's the difference between a curiosity and a live infection rate, and it makes the overall risk easier to underestimate than it should be.

Slopsquatting: how hallucinated package names become live supply chain attack vectors

Seth Larson, Developer-in-Residence at the Python Software Foundation, coined a name for what happens next: slopsquatting. The mechanic is straightforward. An attacker studies which package names LLMs commonly hallucinate, registers those exact names on PyPI or npm, and loads them with a malicious payload. Then the attacker waits, because developers following AI suggestions will eventually try to install the name the model gave them. Names like "aws-helper-sdk" and "fastapi-middleware" have already surfaced in the wild as examples of exactly this pattern.

The 43% repeatability rate is what makes slopsquatting a viable business model for an attacker rather than a lottery ticket. They don't need to guess at a plausible-sounding name and hope; they can observe which fictions a model produces reliably and register accordingly, with reasonable confidence someone will come looking for that exact string. Compounding the problem, 58% of developers, per the same 2025 SQ Magazine compilation, trust AI output without independently testing it, meaning the one human checkpoint that might catch an unfamiliar package name is routinely skipped.

Slopsquatting is a different animal from classic typosquatting. Typosquatting relies on a real package existing, with the attacker registering a near-miss spelling to catch a typo. Slopsquatting has no legitimate package to compare against at all; the AI invented a name that never had a real counterpart, and the attacker simply fills a vacuum the model created. That absence of a reference point is what makes it structurally harder to catch. Traditional registry monitoring looks for new packages that resemble known, trusted ones. Slopsquatting doesn't produce a resemblance to flag. It produces a novelty that looks, on the surface, exactly as legitimate as any other new package on the registry.

The broader supply chain context: AI-generated dependencies enter an already-stressed ecosystem

None of this is happening in a clean environment. Supply chain attacks grew 40% year over year in 2025, and Sonatype identified more than 454,600 new malicious packages that same year, pushing the cumulative total past 1.2 million. That figure alone says something important: the public registry ecosystem is not a trusted baseline waiting to be corrupted. It's already seeded with malicious code at a volume no manual review process could realistically keep up with.

More than 70% of organizations reported at least one supply chain incident tied to third-party software in 2025, and the global cost of these attacks reached an estimated $60 billion that year, with projections putting it at $138 billion by 2031. AI-generated code doesn't create this problem; it compounds one that already existed. Developers following AI suggestions are drawing from a pool that mixes hallucinated packages with real, malicious ones, and nothing in the standard workflow automatically tells them which category they're looking at.

The pattern isn't confined to language-level packages, either. Public container images, including the ones referenced in AI-generated Dockerfiles, typically ship with 50 to 60 known CVEs baked in as a matter of course. An AI-generated deployment manifest that points to an unverified base image carries that risk forward without ever surfacing it to whoever approved the deployment.

CI/CD pipelines carry the same exposure. Attackers have modified historical Git tags on a widely used GitHub Action, one referenced by an estimated 23,000-plus repositories, to exfiltrate secrets through Actions logs. An AI-generated pipeline configuration that pins to a mutable tag rather than a fixed commit SHA is vulnerable to precisely that class of attack, whether or not anyone notices the pin was never locked down. The question isn't whether the ecosystem carries risk; that's established. The question is whether provenance tooling gets applied early enough to catch the problem before it reaches production.

What SBOMs actually capture and where they stop short for AI-generated dependencies

A Software Bill of Materials is a machine-readable inventory: every component, library, and dependency in a piece of software, along with version numbers, license terms, and stated origin. It has moved from a nice-to-have into a compliance requirement, driven by regulatory and procurement requirements across multiple jurisdictions.

But an SBOM has a structural ceiling. Standards like SPDX and CycloneDX describe what's inside an artifact; they don't offer cryptographic proof of how that artifact got built or distributed. An SBOM answers "what's in here." It doesn't answer "which registry actually served this," or "was the build process isolated from outside interference." For AI-generated dependencies specifically, that's precisely the gap that matters: the package might show up correctly in the inventory while its actual path from source to binary remains completely unverified.

Compounding the issue, most SBOMs get generated at the tail end of a build, filed away for compliance, and never referenced again in the workflow where dependency decisions actually happen. It's a snapshot, not a control. Practitioners are beginning to argue that SBOMs should flag which sections of a codebase were AI-generated and by which tool, a category that existing SBOM standards have not yet fully addressed. The model itself has become a third-party dependency, and practitioners are beginning to argue it deserves comparable documentation.

SBOMs capture what exists inside an artifact. They say nothing about whether the build path that produced it can be trusted, which is where SLSA and Sigstore pick up the thread.

How SLSA and Sigstore extend provenance from inventory to verified build path

SLSA, Supply-chain Levels for Software Artifacts, is a framework of progressive levels of build integrity designed to establish verifiable trust in how software is produced. The framework covers a Build track running from L0 through L3, with ongoing work to extend provenance coverage to source as well, as adoption grows across major hosting and publishing platforms.

Signing and transparency tooling complements SLSA on the verification side, and the practice of enriching SBOM entries with repository URLs, commit SHAs, and build attestations turns a static inventory into something that can actually be traced, link by link, back to source.

But attestation is not the same as safety, and a documented incident from May 2026 makes the distinction concrete. In what's been called the "Mini Shai-Hulud" npm supply chain attack, attackers chained together a GitHub Actions misconfiguration, cache poisoning, and OIDC token theft to publish malicious packages under otherwise legitimate developer identities. The compromised packages carried cryptographically valid SLSA build provenance attestations. The signatures checked out. What failed was the build platform itself, which didn't meet SLSA Build L3 isolation requirements, the level specifically designed to prevent this kind of interference during the build. A platform that met L3 would have blocked the attack's primary vector. The lesson is blunt: an attestation is only as trustworthy as the isolation of the system that produced it, and the entire point of SLSA's tiered levels is to make that gap visible rather than assumed away.

Adoption isn't frictionless, either. Community feedback around SLSA implementation has surfaced recurring barriers: incomplete toolchain support, specifications developers find ambiguous, and plain organizational inertia. And SLSA plus Sigstore, even working as designed, verify packages that already exist and were built through a known path. They offer no defense against a hallucinated name that hasn't been registered yet, or one an attacker just claimed an hour ago.

Where provenance breaks down at each stage of an AI-assisted development workflow

Break an AI-assisted workflow into its stages and the provenance gap shows up at every one of them, compounding rather than resetting.

At suggestion, the AI tool proposes a dependency name with no registry lookup, no hash check, no provenance record generated in that moment. The risk here is straightforward: a hallucinated name, a mismatched version, or a real-but-unvetted package enters the developer's head as something already approved, simply because the model said it with confidence.

At install, the developer runs the command and the name resolves, or doesn't, against whatever registry the tool defaults to. If the name was hallucinated and an attacker already claimed it, installation succeeds silently, no warning, no friction. If it doesn't resolve, the developer often goes looking for "something similar," which tends to make the provenance trail worse rather than better.

At build, the dependency gets compiled or bundled into the final artifact. Without hash-pinned lockfiles committed to source control, the build can resolve to a different version than whatever got tested locally, and AI-generated build configurations frequently skip that lockfile hygiene altogether.

At CI/CD, the pipeline runs, often in an environment that doesn't match the developer's local machine. Environment drift means a vulnerability visible in one setting can be invisible in the other. Veracode's 2025 analysis found that 45% of AI-generated code contains security vulnerabilities, and pipeline configuration isn't somehow exempt from that number. An AI-generated pipeline that pins a GitHub Action by mutable tag instead of commit SHA reproduces exactly the class of attack seen in the 23,000-repository incident described earlier.

At SBOM generation, the inventory gets produced at the end of the build, after every prior decision is already locked in. By this point the provenance gap is baked into the artifact. The SBOM records what got used; it cannot retroactively verify where any of it actually came from.

A sixth stage is emerging fast: agentic workflows built on the Model Context Protocol, where AI agents select and install dependencies autonomously through MCP server integrations, with no human in the loop at the moment of decision. A large number of MCP servers have launched on GitHub, and the MCP ecosystem has matured faster than the security guardrails around it. An agent that can install a package on its own can introduce an unverified dependency faster than any human review process, however well designed, has a chance of tracking.

What a provenance-integrated workflow looks like in practice, and what tooling supports it

The failure pattern across every stage above has one thing in common: provenance gets checked, if it gets checked at all, after the decision is already made. A workflow that actually closes the gap has to invert that. Verification needs to happen at the moment of dependency selection, not appended afterward as an audit step nobody has time to act on.

That means a registry lookup and existence check running the instant an AI tool proposes a name, before it ever reaches an install command. It means hash-pinned lockfiles as a default, not an optional discipline left to individual developer habit. It means SLSA provenance attestations checked at build time, with the platform's isolation level treated as a real gate rather than a decorative badge, precisely because the "Mini Shai-Hulud" incident showed a valid signature can coexist with a compromised build path. It means CI/CD configurations that pin actions by commit SHA as a rule, not a suggestion. And for agentic and MCP-based workflows, it means sandboxing and audit logging built into the protocol layer itself, rather than bolted on by whichever team happens to notice the gap first.

None of this eliminates the underlying problem, which is that a language model has no native concept of a verified build path and never will unless that verification gets built around it. But treating provenance as a live, upstream check, rather than a compliance artifact filed away after the fact, is the difference between catching a slopsquatted package before it ships and finding out about it after a production incident. Given how fast AI-generated code is scaling relative to the tooling meant to secure it, that difference is the entire ballgame.