Features¶

pip-compile-multi supports many options to customize compilation. Each option can be specified in requirements configuration file, by replacing dashes with underscores. For example, –use-cache becomes use_cache. Supported configuration files: pyproject.toml, requirements.ini, setup.cfg, tox.ini.

In INI files, use section name starting with requirements:, for example: [requirements:Python 3]

In pyproject.toml prefix section name with tool.requirements, for example: [tool.requirements.python3] or [tool.requirements].

Requirements Directory¶

While it’s a common practice to put requirements files inside requirements directory, it’s not always the case. The directory can be overridden with this option:

-d, --directory TEXT   Directory path with requirements files

In configuration file, use directory option. For example, to use project root, specify:

[requirements]
directory = .

Requirements Files Extensions¶

By default pip-compile-multi compiles *.txt from *.in files. While this is a common naming pattern, each project can use it’s own:

-i, --in-ext TEXT      File extension of input files
-o, --out-ext TEXT     File extension of output files

In configuration file, use in_ext and out_ext options. For example:

[requirements Hashed]
in_ext = txt
out_ext = hash

Disable upgrades¶

When new dependencies are added it’s tempting to keep everything else the same. To recompile .txt keeping satisfying version use --no-upgrade:

--upgrade / --no-upgrade    Upgrade package version (default true)

The option has no effect if there are no existing .txt files.

When using the requirements command, disabling upgrades means running requirements lock. Upgrades are enabled when running requirements upgrade.

Upgrade only selected packages¶

To upgrade only one package and keep everything else untouched, use following option:

-P, --upgrade-package TEXT  Only upgrade named package.
                            Can be supplied multiple times.

When using the requirements command, to upgrade only selected packages pass the list of packages to requirements upgrade command:

requirements upgrade Django cryptography

Under the hood it uses the same option of pip-compile and runs compilation only for files that have one of the passed packages.

This option implies --no-upgrade and takes precedence over --upgrade.

Thanks to Jonathan Rogers.

Use Cache¶

By default pip-compile-multi executes pip-compile without --rebuild flag. --rebuild flag clears any caches upfront and rebuilds from scratch. Option --no-use-cache adds --rebuild flag.

--use-cache / --no-use-cache    Use pip-tools cache to speed up compilation
                                (default true)

In configuration file, use use_cache option:

[requirements]
use_cache = False

Note

But if you run into “magical” issues, try rerunning compilation with --no-use-cache.

Compatible Releases¶

PEP-440 describes compatible release operator ~=. Sometimes it’s useful to have some of the dependencies pinned using this operator. For example, rapidly changing internal libraries. The format for this option is:

-c, --compatible TEXT

where TEXT is a glob pattern for library name. This option can be supplied multiple times.

In configuration file, use compatible option with comma-separated list of package names:

[requirements]
compatible = package1, package2

Generate hashes¶

Put package hash after pinned version for additional security. Format for this option is

-g, --generate-hashes TEXT  Input file name (base.in, requirements/test.in, etc)
                            that needs packages hashes.
                            Can be supplied multiple times.
                            For backwards compatibility can be short
                            environment name (base, test, etc.)

In configuration file, use generate_hashes option with comma-separated list of paths:

[requirements]
generate_hashes = requirements/base.txt, requirements/docs.txt

Example invocation:

$ pip-compile-multi -g requirements/base.txt -g requirements/docs.txt

Example output:

pip-tools==1.11.0         --hash=sha256:50288eb066ce66dbef5401a21530712a93c659fe480c7d8d34e2379300555fa1         --hash=sha256:ba427b68443466c389e3b0b0ef55f537ab39344190ea980dfebb333d0e6a50a3
first==2.0.1         --hash=sha256:3bb3de3582cb27071cfb514f00ed784dc444b7f96dc21e140de65fe00585c95e         --hash=sha256:41d5b64e70507d0c3ca742d68010a76060eea8a3d863e9b5130ab11a4a91aa0e         # via pip-tools

pip requires all packages to have hashes if at least one has it. pip-compile-multi will recursively propagate this option to all environments that are referencing or referenced by selected environments.

Allow Unsafe Packages¶

If your project depends on packages that include setuptools or other packages considered “unsafe” by pip-tools and you still wish to have them included in the resulting requirements files, you can pass this option to do so:

-s, --allow-unsafe          Whether or not to include 'unsafe' packages
                            in generated requirements files. Consult
                            pip-compile --help for more information

In configuration file, use allow_unsafe option:

[requirements]
allow_unsafe = True

This is commonly used with –generate-hashes to avoid generating requirements files which cannot be installed.

Custom Header¶

pip-compile-multi adds a brief header into generated files. Override it with

-h, --header TEXT      File path with custom header text for generated files

In configuration file, use header_file option:

[requirements]
header_file = requirements/header.txt

Limit input files¶

By default pip-compile-multi compiles all .in files in requirements directory. To limit compilation to only a subset, use

-t, --only-path TEXT        Compile only for passed input file paths and
                            their references.
                            Can be supplied multiple times.

For example, to compile one file under Python2.7 and another under Python3.6, run:

$ pip-compile-multi -t requirements/deps27.in
Locking requirements/deps27.in to requirements/deps27.txt. References: []
$ pip-compile-multi -t requirements/deps36.in
Locking requirements/deps36.in to requirements/deps36.txt. References: []

When using requirements command, use include_in_paths configuration option with a comma-separated list of paths. For example:

[requirements Python 3.6]
include_in_paths = requirements/deps36.in

Add index URL annotation¶

Control addition of --index-url options to the output files. Corresponds to pip-compile’s --emit-index-url / --no-emit-index-url flag.

The URL can be defined in the input file as in this example:

--index-url=https://pypi.tuna.tsinghua.edu.cn/simple/
six
click

This option is similar to extra_index_url. The difference is that URL is saved in the output files, which means that it doesn’t need to be configured in every environment.

--annotate-index / --no-annotate-index
                                Add index URL to generated files
                                (default false)

In configuration file, use annotate_index option:

[requirements]
annotate_index = True

Note: the default behavior is not to add the index, i.e., --no-annotate-index.

Add additional index URL to search¶

Pip accepts URLs for additional package indexes through --extra-index-url. The same option can be passed to pip-compile-multi during compilation.

This option is similar to annotate_index. The difference is that the passed URL is not saved in output files, which is helpful if the URL contains private credentials.

--extra-index-url TEXT          Add additional package index URL to search
                                for package versions. Can be supplied
                                multiple times.

In configuration file, use extra_index_url option with comma-separated list of paths:

[requirements]
extra_index_url = https://pypi.acme.com/simple

Warning

Using --extra-index-url option to search for packages that are not in the main repository (such as private packages) is unsafe, per a security vulnerability called dependency confusion: an attacker can claim the package on the public repository in a way that will ensure it gets chosen over the private package.

Use the --index-url option in pip’s configuration file or command line to specify the feed, overriding the default. Avoid the --extra-index-url option, which is additive and may lead to having multiple indexes.

See pip documentation for details.

Add trusted host annotation¶

Control addition of trusted hosts for index URLs to generated files. Trusted hosts can be defined in pip.conf file, input requirements file, or through an argument to pip command. Trusted hosts can have invalid HTTPS certificate, or use unencrypted HTTP protocol.

By default, trusted hosts are saved in the generated files. Pass --no-emit-trusted-host to remove it.

If no trusted hosts are defined, this flag doesn’t have any effect.

--emit-trusted-host / --no-emit-trusted-host
                                Add trusted host to generated files
                                (default true)

In configuration file, use emit_trusted_host option:

[requirements]
emit_trusted_host = False

See also: annotate_index and extra_index_url options.

See pip-tools issue #382 for more details.

Autoresolve cross-file conflicts¶

Compile requirements file, that references all other files first, and than use it as a constraint.

--autoresolve/--no-autoresolve Automatically resolve
                               cross-file conflicts.

When using requirements command, this option is enabled by default. To disable it, set it to False in a configuration file:

[requirements]
autoresolve = False

This strategy allows to resolve cross-file conflicts of two types:

Files FOO and BAR both have dependency PKG, but compile it to different versions.
FOO has PKG resolved to version 3. BAR references FOO, but also has another dependency, that constraints PKG to version 2.

By compiling all-including file (aka sink), that references both FOO and BAR first, pip-compile-multi generates conflict-free set of versions.

After that, this compiled file is passed as a constraint for compiling all requirements files.

As the last step, the sink file is compiled again preserving reference files and skipping duplicate packages.

Note

This feature works only if your project has a single requirements file, that references (directly or indirectly) all other files.

Backtracking resolver¶

Pip has an option to enable backtracking conflict resolution logic, which can automatically downgrade some dependencies to meet constraints from other packages. See also the a note on resolvers in pip-compile project.

--backtracking / --no-backtracking
                            Enable backtracking resolver. Translates to
                            pip-compile --resolver=backtracking option.

In configuration file, use backtracking option:

[requirements]
backtracking = True

Skip constraints in comments of output files¶

When input files contain constraint references (e.g. ‘-c constraints.in’), pip-compile adds it to “via” comments. For example:

rsa==3.4.2
     # via
     #   -c constraints.txt
     #   google-auth

When this option is enabled (default) that snippet will be converted to:

rsa==3.4.2
     # via google-auth

Saving two lines in .txt file. This feature is especially useful in combination with Autoresolve cross-file conflicts.

--skip-constraints / --no-skip-constraints
                              Remove constraints from "via" comments.

To include constraint comments, set it to False in a configuration file:

[requirements]
skip_constraints = False

Strip extras¶

Instructs pip-compile to attempt to omit extras in transient dependencies, while assuring the constraints compatibility.

--strip-extras          Try avoiding use of extras.

In configuration file, use strip_extras option:

[requirements]
strip_extras = True

Live output¶

Print debug output from pip-compile live. If the option is disabled (by default) the debug output is printed only in case of failure.

--live / --no-live              Print debug output from pip-compile live.

In configuration file, use live option:

[requirements]
live = True

Enable UV¶

UV is an extremely fast Python package installer and resolver written in Rust. When enabled, pip-compile-multi will use uv’s dependency resolver instead of pip-tools.

--uv / --no-uv      Use uv for dependency resolution.

In configuration file, use uv option:

[requirements]
uv = True

Key differences between uv and pip-tools output:

UV is significantly faster at dependency resolution. Particularly noticeable in large projects with complex dependency trees.
UV’s resolver is more aggressive at finding newer versions.

To use UV:

Install uv (pip install uv)
Pass --uv flag to pip-compile-multi or add uv = True when using requirements command.

Check that `pip-compile-multi` was run after changes in `.in` file¶

pip-compile-multi adds a special line (before header) at the beginning of each generated file. This line contains a SHA1 hash of the .in file’s contents.

Command

$ pip-compile-multi verify
OK - requirements/base.txt was generated from requirements/base.in.
OK - requirements/test.txt was generated from requirements/test.in.
OK - requirements/local.txt was generated from requirements/local.in.
OK - requirements/testwin.txt was generated from requirements/testwin.in.

Or, if using requirements command:

$ requirements verify
OK - requirements/base.txt was generated from requirements/base.in.
OK - requirements/test.txt was generated from requirements/test.in.
OK - requirements/local.txt was generated from requirements/local.in.
OK - requirements/testwin.txt was generated from requirements/testwin.in.
OK - requirements/base.hash was generated from requirements/base.txt.
OK - requirements/test.hash was generated from requirements/test.txt.
OK - requirements/local.hash was generated from requirements/local.txt.
OK - requirements/testwin.hash was generated from requirements/testwin.txt.

recalculates hashes for .in files and compares them with the stored values.

If verification fails, an error message is logged and exit code 1 is returned:

$ pip-compile-multi verify
ERROR! requirements/base.txt was not regenerated after changes in requirements/base.in.
Expecting: # SHA1:7d82ce5a82b0a6cf91b2c4debe90eb1e5ef37f37
Found:     # SHA1:32737333f763ceffd22b7fcb76fbe62a538296fa
OK - requirements/test.txt was generated from requirements/test.in.
OK - requirements/local.txt was generated from requirements/local.in.
OK - requirements/testwin.txt was generated from requirements/testwin.in.

In big teams it might be a good idea to have this check in tox.ini:

[testenv:verify]
skipsdist = true
skip_install = true
deps = pip-compile-multi
commands = pip-compile-multi verify
whitelist_externals = pip-compile-multi

Features¶

Requirements Directory¶

Requirements Files Extensions¶

Disable upgrades¶

Upgrade only selected packages¶

Use Cache¶

Compatible Releases¶

Generate hashes¶

Allow Unsafe Packages¶

Custom Header¶

Limit input files¶

Add index URL annotation¶

Add additional index URL to search¶

Add trusted host annotation¶

Autoresolve cross-file conflicts¶

Backtracking resolver¶

Skip constraints in comments of output files¶

Strip extras¶

Live output¶

Enable UV¶

Check that `pip-compile-multi` was run after changes in `.in` file¶

pip-compile-multi

Navigation

This Page

Features¶

Requirements Directory¶

Requirements Files Extensions¶

Disable upgrades¶

Upgrade only selected packages¶

Use Cache¶

Compatible Releases¶

Generate hashes¶

Allow Unsafe Packages¶

Custom Header¶

Limit input files¶

Add index URL annotation¶

Add additional index URL to search¶

Add trusted host annotation¶

Autoresolve cross-file conflicts¶

Backtracking resolver¶

Skip constraints in comments of output files¶

Strip extras¶

Live output¶

Enable UV¶

Check that pip-compile-multi was run after changes in .in file¶

Check that `pip-compile-multi` was run after changes in `.in` file¶