How to share code between Python internals projects?

Question

Let's say I am building an event-driven architecture, with Python microservices. I have a stream package handling all interactions with the streaming platform. It is supposed to be used internally only, in the company.

stream
    __init__.py
    produce.py
    process.py
    security.py

What are my options for sharing this package between all my Python microservices?

Should I publish my stream package so my projects can install it?

Is there some kind of Gradle for python including the Multi project feature?

Answer 1

You can package your Python code to reusable packages.

Poetry is a popular modern tool to manage your Python package.

Poetry, and other Python package managers like pip, can directly install private packages from an internal file server or Git link.

Answer 2

You don't have to publish your Python modules to install them with pip. See Packaging Python projects from the Python documentation to understand how to create installable packages (Wheel, .whl) from your project and stop once you've got the .whl file (don't have to upload to the public package index). Then, you can put them either into your company-internal Python package index (e.g., artifactory) or into an object storage service or Git LFS, from where your other microservices can access and install the package during build with

pip install /path/to/your/stream-0.0.1-py3-none-any.whl