Python namespace packages

Question

I'm following this instructions to create multiple small (independent) Python2 packages from a big one - https://packaging.python.org/guides/packaging-namespace-packages/#pkg-resources-style-namespace-packages

Now I have multiple sub(packages) with such a structure: https://github.com/pypa/sample-namespace-packages/tree/master/pkg_resources

I can install and use them independently, it works just perfect. But since I have 12 packages under the same namespace, I would like to be able to install them by running one command (e.g. pip install). For example. some future projects will need them all, so I prefer to serve them as (regular) monolith (one dependency), but sometimes I need only one of them (that's why I play with namespaces). so I need some setup.py file in the root directory of my namespace that contains all subpackages in install_requires or what? I can't figure out how can I have an option to install all my small packages at once under the root namespace like in any regular project structure (when we don't split them and don't use namespaces at all), but having an option to install them separately?

Answer 1

This sounds like a general problem of how to manage dependencies correctly, which is not related to your shared namespace feature. For your special problem, I would recommend the following:

install_requires is only for the absolute necessary dependencies (those which needs to be present or the program cannot be executed), so it does not fit for your case.

It is better to use the extras_require directive, which allows you to specify additional dependencies in certain cases. Common extras are dev – packages needed during development – or doc – packages required for building docs.

Assuming your packages which are under the same namespace are called nsp1, nsp2, and nsp3, you could specify in any of your packages in your setup.py:

setup(
    name="Project-A",
    ...
    extras_require={
        'all':  ["nsp1", "nsp2", "nsp3"],
        'set1': ["nsp1", "nsp2"],
    }
)

Afterwards, you can install them with pip3 install nsp1[all] (assuming you have extended the setup.py for nsp1) or pip3 install .[all] for local install.

If any of your “some future projects” needs exactly those dependencies, you can add those extras to its dependencies, e.g. in its setup.py:

install_requires=['nsp1[all]', …]

An alternative would be to use a requirement.txt:

nsp1
nsp2
nsp3

And install them with pip install -rrequirement.txt