Container Image Reproducibility and Provenance Attestation

Container image reproducibility and provenance attestation address two different failure modes in the software supply chain, and neither is sufficient on its own. A bit-for-bit reproducible image proves that two artifacts are identical; it says nothing about whether the source was legitimate or the pipeline that built it was trustworthy. The rest of this piece works through why reproducibility fails more often than developers assume, what closes that gap, and how organizations are implementing both properties together today.

Why container images are not reproducible by default

Developers who pin their dependency versions tend to assume the problem is solved. A 2026 study titled "It's Not Just Timestamps: A Study on Docker Reproducibility" rebuilt images from a stratified sample of GitHub repositories and found that only a small fraction of buildable images were bitwise reproducible without infrastructure changes, and even after those changes were applied, the majority of buildable Dockerfiles remained non-reproducible. The title itself makes the point: timestamps are the easy, visible cause that gets fixed first, and fixing them does not come close to solving the problem.

The study's deeper finding is that non-reproducibility is structural rather than a matter of sloppy Dockerfile hygiene. Developers routinely pin their top-level packages but leave transitive dependencies unpinned, so a silent upstream update to a sub-dependency changes the build environment without anyone touching the Dockerfile. Beneath the application layer, a third mechanism operates: package managers inside the container ignore the state of the underlying operating system, so when a developer's laptop and a build server carry different versions of the C libraries a package needs to compile against, the build fails unpredictably or, worse, succeeds with different binary output.

This is not a tail-risk phenomenon confined to unusual projects. Separate research on flaky build behavior found that a notable share of Dockerfiles drawn from candidate projects exhibited flaky behavior, and because that sample skewed toward high-quality, well-maintained projects, the real-world rate across the broader ecosystem is almost certainly higher. The SolarWinds incident is the reference case the research behind this argument points to: it did not involve container registries specifically, but it is the clearest illustration of how a compromised build or publication pipeline can distribute tampered software to a wide population of trusting consumers.

Reproducibility in Practice

Reproducibility is not a vague aspiration. Given the same source code and the same build instructions, any independent builder should produce an image whose SHA-256 digest matches exactly, and that digest, a content-addressable hash of the image's contents, is the entire test. There is no partial credit: either the hash matches or it does not, and if it does not, something in the build environment, the dependency graph, or the build process introduced nondeterminism somewhere along the way.

The techniques that close the gap map directly onto the root causes the research identifies. Operating system packages need to be pinned to exact versions, and language-level dependencies should be hash-verified using tools such as pip-compile rather than left to resolve against whatever version happens to be available at build time. None of this is exotic engineering. The guidelines the research derives, pinning dependencies at every layer, cleaning build artifacts, eliminating floating version ranges, translate directly into linter rules and CI checks that a team can enforce automatically rather than relying on documentation that developers may or may not read.

The Freedom of the Press Foundation's Dangerzone project shows that this is achievable in a real, security-sensitive production system rather than only in a research paper. Dangerzone treats reproducibility as a prerequisite rather than a bonus feature, and as of version 0.10.0, any user can build the project's container image from a specific git commit and obtain a bit-for-bit identical copy of the image the project itself signed and distributed. That result, an independent party reproducing the exact artifact a development team published, is what the techniques above are built to deliver.

Where reproducibility stops: image origin and build process

Reproducibility, however rigorously implemented, proves only that two artifacts are identical. It says nothing about whether the build process that produced them was trustworthy, whether the source code was legitimate, or whether the pipeline was hardened against tampering at any stage. Reproducibility answers "are these two things the same," and that is a different question from "should anyone trust this thing in the first place."

Image tags compound the problem rather than solving it. Because a tag is a mutable reference that can be reassigned to point at an entirely different image at any time, any trust claim attached to a tag rather than to the underlying SHA-256 digest is fragile by construction. A team that verifies an image once at a given tag has no guarantee that the same tag points to the same bits a week later.

Answering where an image actually came from requires a distinct set of questions that reproducibility was never designed to address: which repository and commit the image was built from, which workflow or pipeline step produced it, what builder identity ran the build, whether the build environment was isolated enough that one run could not influence another, and whether the build timestamp is tamper-evident rather than simply asserted. The EU Cyber Resilience Act requires software makers to create and maintain software bills of materials in a standardized, machine-readable format, and a national government's executive order and subsequent policy memos push SBOMs and secure development practices for federal suppliers.

An SBOM lists the inventory of components inside an image. It does not answer the pipeline integrity questions above, and generating the SBOM itself is the easy part: whether it does any good depends on the governance, scanning, and remediation workflows needed to act on what the SBOM reveals. Reproducibility establishes that an artifact is internally consistent. Provenance attestation is the mechanism that establishes where that artifact came from and whether the process that produced it can be trusted, and the two properties are complementary rather than substitutes for one another.

The signing stack: from Sigstore to SLSA Level 3

Provenance attestation is a cryptographically signed, tamper-evident record anchored to the image's digest. Anything less than a cryptographic anchor can be altered after the fact without leaving a trace, which defeats the purpose of having a record.

The signing stack that makes this work rests on Sigstore's keyless signing model, which combines a Fulcio certificate establishing build identity with an entry in the Rekor transparency log that provides tamper-evident timestamping. The anchor for all of this is the SHA-256 digest of the image, never the tag, since the tag remains a mutable pointer that offers no guarantee about what it currently references.

Before reproducing or signing, Dangerzone's own builders verify the attestation using a CUE policy that checks both the builder identity, matching against the slsa-github-generator container workflow, and the workflow entrypoint, matching the exact path .github/workflows/release-container-image.yml, along with the source repository URI. That policy is a concrete answer to the "which repo, which workflow, which identity" questions raised earlier, expressed as machine-checkable code rather than as a claim a team simply asserts and hopes reviewers accept.

Level 3 is the target most regulated programs aim for because it is the first level where provenance is non-falsifiable by a malicious build step.

Implementing SLSA Level 3 in practice: Tekton Chains and Red Hat Konflux as a reference architecture

SLSA Level 3 is achievable today using open components, and production deployments show where the real difficulty lies: the isolated builder requirement, not the cryptographic signing, is the hardest part of the architecture to get right. Guaranteeing that no build step can influence another, or forge its own provenance, requires rethinking how the build platform itself is structured.

A Kubernetes-native reference architecture for Level 3 combines several components, each addressing one part of the requirement. Tekton Chains sits alongside it and handles provenance generation automatically: when a build job completes successfully, Chains captures the build's inputs and outputs and generates a signed attestation without any user-written script producing that record. Sigstore, through Fulcio and Rekor, supplies the keyless signing and transparency logging that anchors the attestation cryptographically, and Cosign handles verification at pull time, letting any consumer check the attestation before trusting the image.

The resulting provenance document includes the builder ID, the build definition, external parameters such as the git URL and commit, and the image digest as its subject. A third party can verify the full chain without needing to trust the team that actually built the image. Red Hat Konflux, built on Tekton and using Tekton Chains for signed provenance, is a production example of this architecture running at scale: completed build jobs carry a signed Level 3 provenance attestation that any third party can verify independently. The architectural constraint that makes all of this non-falsifiable is that provenance has to be generated by the build platform itself, Chains running as a cluster controller outside any individual build's control, rather than by a script the user supplies as part of the build.

Dangerzone's "Triangle of Secure Code Delivery

The Freedom of the Press Foundation's Dangerzone project demonstrates that reproducibility and provenance attestation function as sequential checks within a single trust chain rather than as competing approaches to the same problem. Because the container doing that conversion handles deliberately untrusted, potentially malicious input, the integrity of the container image itself is part of the attack surface: a tampered image could be used as a path to escape the sandbox.

Dangerzone calls its answer to this problem the "Triangle of Secure Code Delivery," a three-part sequence: attest provenance, verify reproducibility, sign. The order is deliberate rather than incidental. Signing comes last, sealing the result of both checks into a record other parties can verify without repeating the work themselves.

GitHub runners add SLSA attestations to the container registry at the moment of build, and Dangerzone does not take those attestations on faith. Before reproducing or signing anything, Dangerzone's builders verify the attestation against the CUE policy described earlier, checking the builder identity, the workflow entrypoint, and the source repository URI all at once. That step means the project is not blindly trusting that a given GitHub runner is who it claims to be: the attestation functions as a machine-checkable claim about the runner's identity, and the CUE policy is what actually checks it rather than assumes it.

The independent container update feature shipped in version 0.10.0 was built so that users never have to extend blind trust to the distribution registry itself. Any user can independently verify all three properties in the triangle on their own machine, without taking Dangerzone's word for any of them. Signing throughout this process runs on the Sigstore and Cosign ecosystem, and the cosign verify-attestation command handles the entire verification chain on the end user's side, collapsing what would otherwise be a multi-step manual audit into a single command. What the Triangle demonstrates is that skipping any one of its three steps breaks the guarantee the other two are meant to provide: reproducibility without provenance verifies the wrong artifact, and provenance without signing leaves the attestation itself unprotected against tampering after the fact.

Provenance attestation in security tooling: the Snyk GA release

Provenance attestation has moved from a framework that security-focused teams assemble themselves into a feature available inside commercial container security tooling. Snyk reached general availability on container image provenance attestations on September 29, 2026, reading provenance from BuildKit-style OCI attestations as its source format. That detail matters because it signals the format has standardized enough that a commercial scanner can consume it directly rather than requiring a custom integration for each customer's pipeline.

Inside Snyk's container image inventory, the feature surfaces the source repository and commit, the builder identity and build type, the build timestamp, and the Dockerfile contents. Snyk's documentation describes two use cases for this data. The second is remediation routing, where the source repository and commit data captured in the attestation lets a security team route a discovered vulnerability directly to the team that owns the build that introduced it, rather than spending time tracing an image back to its origin by hand.

That a mainstream commercial scanner now treats provenance attestation as a standard inventory feature, alongside vulnerability scanning and SBOM generation, confirms that provenance has become an operational expectation for container security programs rather than a specialized practice reserved for projects like Dangerzone or platforms like Konflux. The techniques described throughout this piece, reproducible builds verified against a digest and provenance attestations anchored to a signed, non-falsifiable record, are converging into the baseline a serious container security program is expected to meet.