How do I freeze the requirements of a tox test environment?

Question

I usually install my requirements from an abstract file like so:

# requirements.in
pytest

I define test environments in a tox.ini like so:

[testenv]
deps = -rrequirements.in
commands = pytest

How do I freeze the installed versions of the dependencies to a requirements.txt file?

I tried using this bash one-liner:

commands_post = python -m pip freeze --all > requirements.d/pytest.txt

However, this prints the requirements to the standard output instead of redirecting them to the file:

pytest-py312: commands_post[0]> python -m pip freeze --all '>' requirements.d/pytest.txt

Note that the > is getting escaped instead of being interpreted.

Answer 1

One solution to this is to allow using bash and run the pip freeze command inside bash:

[testenv]
allowlist_externals = bash,/bin/bash,/usr/bin/bash
commands = pytest
commands_post = bash -c "python -m pip freeze --all > requirements.d/pytest.txt"
deps = -rrequirements.in

Note: This solution is not portable to systems without bash.

Answer 2

You should use pip-tools to freeze your requirements.

https://github.com/jazzband/pip-tools

A suitable tox env could look like this https://github.com/jugmac00/flask-reuploaded/blob/0b050952856ed28b1d5f534c951c07bf7dabbeff/tox.ini#L72

Answer 3

One could use Python's subprocess.check_output function:

[testenv]
deps = -rrequirements.in
commands = pytest
commands_post = python -c "import subprocess; open('requirements.d/pytest.txt', 'wb').write(subprocess.check_output(['python', '-m', 'pip', 'freeze', '--all', '--exclude-editable']))"

NOTE: This copies the requirements from C to Python memory and back to C for writing them to disk. This is cumbersome and slow.

Answer 4

One could use pip's internal API:

[testenv]
deps = -rrequirements.in
commands = pytest
commands_post = python -c "from pip._internal.operations.freeze import freeze; open('requirements.d/pytest.txt', 'w').write('\n'.join(freeze(exclude_editable=True, skip=())));"

Note: Using pip's internal API is discouraged. So proceed with caution.

Answer 5

One could use Python's subprocess.run function:

[testenv]
deps = -rrequirements.in
commands = pytest
commands_post = python -c "import subprocess; subprocess.run(['python', '-m', 'pip', 'freeze', '--all', '--exclude-editable'], stdout=open('requirements.d/pytest.txt', 'w'), text=True)"

Nicer formatting:

commands_post =
    python -c "\
        import subprocess; \
        subprocess.run(\
            ['python', '-m', 'pip', 'freeze', '--all', '--exclude-editable'], \
            stdout=open('requirements.d/pytest.txt', 'w'), \
            text=True,\
        )\
    "

This solution uses the stdout parameter to hand over an output file, which should be fast.