Python Dependency Management in Reproducible Build Systems

Python dependency management in reproducible build systems needs control over how resolution and locking work, which build artifacts they produce, and the environment around them. One-command developer onboarding across macOS, Linux, and virtualized Linux environments requires an environment definition that encompasses installation. A versioned, declarative definition must carry the full development stack, while package locks and controlled builds address the layers it depends on.

Why Python dependency management is harder to get right than it looks

“It works on my machine” describes a mismatch. One machine may resolve newer dependencies, another may have different system libraries, and a third may compile the same source into different artifacts. These failures compound because each layer supplies inputs to the next. A correct dependency graph cannot compensate for an uncontrolled compiler or an unauditable binary.

Resolution determines which package versions can coexist, while locking records the selected versions. Isolation separates project dependencies from other projects and the host, and dependency groups keep development tools from leaking into production. Binary reproducibility then asks whether those chosen inputs yield the same artifacts from build to build. Solving resolution without locking permits drift, while locking without isolation leaves room for leakage. The problem therefore calls for coordinated controls rather than one installer expected to govern everything.

How lock files address resolution non-determinism — and where they stop

Without a committed lock file, each laptop, CI runner, staging environment, and production deployment resolves dependencies independently. An upstream release between those operations can change the selected graph even when the project configuration stays unchanged. Committing the lock file removes that repeated choice by recording exact versions. It is the minimum necessary condition for reproducible installs, though it does not establish reproducibility for the entire build.

PEP 751 establishes pylock.toml as a unified format, reducing the dependence between a lock artifact and the tool that generated it. That standard gives installers a common target rather than requiring every workflow to revolve around a proprietary format. Standardization is not, however, evidence that every installer already handles the format identically.

More importantly, a portable version selection remains only a version selection. It does not control timestamps in compiled bytecode, native-extension build flags, or the provenance of downloaded wheels. Different system libraries and architectures can also produce different binaries from the same locked packages.

What the consolidated tooling landscape actually offers for reproducibility

Choosing among uv, Poetry, and pip-tools depends on what the Python tool needs to do for reproducibility. Their existing lock artifacts differ, and PEP 751 offers a direction toward portability. uv emphasizes cross-platform locking and CI setup, Poetry combines locking with project lifecycle management, and pip-tools keeps the workflow close to pip. These are different operating models.

Astral’s Rust-based uv has a universal lock file and also manages the Python version. That combination controls both dependency selection and the interpreter, reducing the opportunity for them to drift independently. Its single binary requires no Python installation to bootstrap, which removes an ambiguity from CI setup. Its pip-compatible interface also lowers migration friction. However, uv retains pip’s limitations for complete source-built dependency trees, and its dependency groups are less sophisticated than Poetry’s arbitrary group system.

Poetry stores exact versions in poetry.lock and has one interface for resolution, virtual environments, building, and publishing. Its selectively installed dependency groups suit teams separating development, test, documentation, and CI requirements. pip-tools compiles pinned requirements.txt files from abstract requirements.in files while leaving full project management to other tools. Its platform-specific output is a trade-off for teams spanning operating systems or architectures. None of these tools pins the system libraries and compilers around it on its own.

The bytecode variability problem and the limits of the --reproducible flag

Python compiles source into .pyc files during installation or import, and timestamp-based files carry metadata that can vary between builds. Identical source can therefore yield different artifact hashes. That variability undermines hash-based deployment verification and reduces the usefulness of cached image layers.

PEP 552 added hash-based .pyc files that can be used in place of timestamp-based files. The important distinction is between compiling bytecode and compiling it deterministically. uv’s --compile-bytecode option enables installation-time compilation, but uv does not expose a --reproducible flag or native PEP 552 hash-based integration. Consequently, uv pip install --reproducible is not a supported CI remedy. Hash-pinned deployment workflows need the deterministic bytecode mechanism.

Why pre-built wheels are insufficient for regulated and high-trust environments

A pre-built wheel delegates part of the trust decision to someone else’s build system. The consumer receives an artifact compiled from source they may not have reviewed, using tools and flags they did not select. That workflow is adequate for many development environments. It is insufficient where financial services, government, defense, or regulated AI requirements demand a documented chain from reviewed source to deployed binary.

Source builds also become necessary when published wheels do not match a required architecture or CUDA or ROCm configuration. Managing x86_64, aarch64, CPU-only, and CUDA variants requires consistent package inputs and controlled variant-specific builds, with binaries allowed to differ across targets.

Simply disabling binary downloads in pip does not solve the entire problem. Build tools have their own build dependencies. The setuptools bootstrap cycle can reintroduce pre-built wheels. pip also lacks the required per-package customization, patching, and coordinated multi-variant pipeline. Nix, Gentoo, and other package managers address source builds through handwritten package specifications, but maintaining those recipes adds work for large, changing Python trees.

How Fromager turns Python dependency trees into auditable, source-built pipelines

Fromager uses published Python build metadata to rebuild dependency trees from source without a handwritten recipe for every package. PEP 518 declarations in pyproject.toml and PEP 517 build metadata provide the requirements that this process follows. The command fromager bootstrap stevedore illustrates the mechanism with the OpenStack plugin management library. Fromager resolves the package, downloads source, resolves build requirements, and builds dependencies in order, repeating the process through the tree.

The resulting build-order.json records the sequence needed to build every package. Its companion, graph.json, exposes the dependency relationships that explain that sequence. Together, they make the pipeline inspectable rather than leaving build order hidden inside a successful installation. These artifacts help provide the audit trail that workflows building from source need.

On Linux, Fromager can run build commands in network namespaces with unshare --net when network isolation is enabled. Those commands have no outbound connectivity, supporting builds without PyPI access. Fromager also manages package builds across hardware variants, which is especially useful for AI/ML trees with a lot of native code. Its role is therefore specific to source construction and provenance, alongside uv or Poetry’s locking workflows.

Software supply chain security and SBOMs as a built-in requirement, not an audit bolt-on

White House memorandum M-26-05, “Adopting a Risk-based Approach to Software and Hardware Security,” issued January 23, 2026, eliminates mandatory federal-level SBOM requirements and leaves adoption to each agency’s discretion. It does not support a claim that every federal supplier faces a universal SBOM mandate. What each agency expects and what high-trust procurement requires still make it important to be able to trace artifacts.

An SBOM can describe a downloaded binary, but it cannot establish a source-to-build provenance chain that was never supplied. Documented attacks involving typosquatting, compromised maintainer accounts, and malicious build hooks make that distinction important. The platform can help through dependency analysis, malicious-package detection, checks on artifact integrity, and enforcement of policy at the release gate.

For AI/ML, the direction is toward extending SPDX and CycloneDX with model metadata and training-data provenance. PyTorch and TensorFlow remain software components, while CUDA and hardware-specific variants also need representation. Flox environments carry input-derived hashes that can be attached to deployments and generate deterministic SBOMs. Those properties make inventory and deployment identity part of the environment workflow, supporting CVE discovery and hash-level blocking or patching rather than reconstructing the environment during an audit.

Environment-layer reproducibility: why isolation must extend from laptops through CI to production

Virtual environments separate Python dependencies, but they do not capture the interpreter, system libraries, environment variables, or tools used to create them. A CI runner can therefore pass a build that fails in production despite using the same package lock.

Interpreter pinning narrows that gap. uv can enforce the Python version recorded in .python-version, within the same workflow. Yet the libraries linked by that interpreter can still differ. Full-stack environment management extends pinning to those external inputs, which Flox treats together with language-level dependencies in a single manifest. The aim is to control the inputs for each supported platform while allowing binaries to differ at the byte level across operating systems.

Containers and declarative environments are complementary. Containers freeze deployment state, but using images as the development primitive can separate developers from local tools and require a build-scan-tag-push cycle for changes. A human-readable manifest.toml instead defines a native environment with atomic edits and rollbacks. Built on Nix, Flox supports that model and can emit minimal, distroless OCI images when deployment requires them. The environment definition remains authoritative while the container serves as a delivery artifact.

How to make one-command developer onboarding an operational test of reproducibility

A new engineer tests whether the environment can actually be reconstructed. Setup instructions that have not been run from scratch often conceal manual steps or entry points that require a guide. A single command backed by an authoritative environment definition removes that dependence on remembered procedure. For teams standardizing across macOS, Linux, and virtualized Linux environments, the criterion is whether the definition covers the full stack, including Python packages. Any uncovered dependency remains a manual setup obligation.

That distinction also determines the cost to platform teams. When setup depends on individual intervention, senior engineers become part of the installation process. With a declarative entry point, onboarding exercises the same environment contract used elsewhere. Flox’s manifest.toml gives developers and agents the same pinned references and rollback capability, making the definition usable by both.

An AI coding agent tests the same property without the benefit of unstated human assumptions. An underspecified environment can produce non-deterministic execution, making drift a correctness problem. The Model Context Protocol lets agents invoke platform actions through exposed tools, including environment operations. That interface still needs the environment contract beneath it, and large multi-tenant deployments still need careful work to harden security and observability. Humans and agents need the same reproducible foundation even when they reach it through different interfaces.

Choosing the right combination of tools for each layer of the reproducibility stack

uv fits new projects and CI-heavy workflows that value cross-platform locking and interpreter pinning. Poetry fits teams needing sophisticated dependency groups and a unified building and publishing workflow. pip-tools offers a smaller change from pip when platform-specific output is acceptable. PEP 751 provides the direction for portable locks without removing these workflow distinctions.

Artifact controls come next. Timestamp-independent bytecode requires the hash-based mechanism introduced by PEP 552. Where reviewed-source provenance is required, Fromager supplies recursive source builds and inspectable ordering and graph artifacts. Neither control can substitute for the other, because one governs artifact variability while the other governs how the binary was produced.

The environment layer then binds package tools to pinned interpreters, system libraries, and build tools. With FloxHub, environments can be versioned and shared by name, pulled on demand, and rolled back by generation. OCI output preserves container deployment workflows without making images the source of the environment definition. Reproducibility is therefore an infrastructure responsibility. It comes from treating the environment as a versioned, declarative artifact at every layer, rather than expecting the right installer to make the whole system repeatable.