python import from different directory doesnt work

Question

Quite new to python (using v3.11). I am trying to import something from another directory. I tried several different ways and the only way i was able to make it work is by adding ABSOLUTE path using sys.path.insert, which is obviously not correct. I suspect it is because im attempting to test class with factory method and I feel like i missed something.

This is what i have:

└── /project
    ├── /src
    │   ├── /services
    │   │   └── service1.py
    │   │         class DataService:
    │   │             def write_data(self,data,location):
    │   └── containers.py
    │             from dependency_injector import containers, providers
    │             from services.service1 import DataService
    │             class Container(containers.DeclarativeContainer):
    │                 data_service = providers.Factory(DataService)
    └── /tests
        └── /services-tests
            └── test_service1.py
                     import pytest
                     from containers import Container #<--- ModuleNotFoundError: No module named 'containers'
                     data_service = Container.data_service()  

As you can see in tests\services-tests\test_service1.py, i am attempting to import, and it fails. As I said, I tried several things, but the only way I got it to work is by adding ABSOLUTE path using sys.path.insert, which is obviously not correct way. Any ideas where I may have gone wrong?

Answer 1

Does python3 -m tests.services-tests.test_service1 work for you when you are in project folder?

Answer 2

This is a scoping issue. While both are under Project, test_service1.py is in the folder project/tests and containers.py is 'out of scope' for it to be simply imported without identifying that it is in a parallel folder project/src.

So, from containers import Container would only work if both Python scripts were in the same folder. You are almost at the final answer when you added the absolute path using sys.path.insert. In many workflows where developers deal with huge repositories of code, they often do this (and this is recommended):

sys.path.append("path/to/src")
from containers import Container

There are dot imports available where a single dot represents the current directory, double dot represent a parent directory, three dots represent grandparent directories and so on. So a rough workaround is

from ...src import containers.Container

This is always 50-50 for me and I prefer the sys.path.append() method everytime. The most I've used is 2 dots so 3 dots is something to try.

Additionally, good practice requires an __init__.py file in the directory to tell python that it is a package and relative imports can be done.

Answer 3

You should add an (empty) __init__.py file in all subdirectories of project. This makes src and tests into packages. The imports should read

from src.services.service1 import DataService # or
from .service1 import DataService # relative import

and

from src.containers import Container

and so on. Both the tests and main script should be run from the project dir using the -m switch.