Can I combine pip installation based on requirements.txt with optional extra packages PEP508?

Question

For some while, I have been looking for a method to use a requirements.txt file (see format specification) with optional package extras depending on the use case, e.g., for testing purposes:

# requirements.txt
pandas >= 2
pytest [test] >= 7
pytest-testdox [test]

I follow the standard definition in PEP 508 for defining and grouping these optional extras.

I would like to use a command like the following one, which does not work:

pip install -r requirements.txt[test]

Ideally, there would be a flag for specifying the extras involved but there is none as far as I know.

Workaround

I was writing in the end a pyproject.toml, which allows the following workaround as proposed in the official documentation:

pip install '.[test]'

This command runs successfully based on the following (probably minimal) file:

# pyproject.toml
[tool.setuptools]
packages = ["myproject"]

[project]
name = "myproject"
version = "0.1"
dependencies = [
    "pandas",
]

[project.optional-dependencies]
test = [
    "pytest >= 7",
    "pytest-testdox",
]

This approach, however, builds a new unnecessary package myproject out of the given file and folder, which I want to avoid.

Answer 1

Configuring your extras in pyproject.toml is the modern way to go.

Simply install your project as editable with -e, and it won't be separately built.

pip install -e '.[test]'

If you do want to use requirements.txt style files and you're using recent versions of setuptools as your build backend, you should be able to specify those files as dynamic metadata sources.

Answer 2

The purpose of the extra specifier is not to control which packages in the requirements.txt to be installed. Instead, it is used to mark the optional dependencies of installed packages. For example, if you specify pytest[test] in your requirements.txt, you will install the extra packages which are used for testing the pytest package.

As far as I know, it's not quite possible to pass parameters to the pip install -r command. An alternative way to achieve this is to create multiple requirement.txt files, or control it with packaging utilities (setup.py or pyproject.toml, etc.).

See also: Optional dependencies in distutils / pip