Python .pth and ._pth files (standard install vs. embedded Python)

Question

I see two diffrent behaviours about .pth and ._pth files:

• the official Python install for Windows uses .pth files in lib\site-packages as documented in https://docs.python.org/3.12/library/site.html#module-site. It seems that adding multiple .pth files works and is used at the Python interpreter startup.

• the Python Embedded package for Windows seems to use ._pth files (and not .pth!) for the same thing: python38._pth, but in the same folder as python.exe. In this case it seems that adding a second .pth or ._pth file in the same folder isn't used by the interpreter. Why? Is it documented that, for embedded Python, only one ._pth will be used?

More generally, what are the rules about .pth vs ._pth files? In which case should we use one or the other, what is the reason for the underscore _pth?

Note: the rules seem here complicated (.pth vs ._pth ; multiple files allowed vs. only one file used).

NB: Documentation about ._pth files

Answer 1

From what I understand of the documentation provided, I see:

• .pth Files: used in standard Python installations to add extra directories to Python's module search path (sys.path).
  The site module processes these files at startup, allowing for the dynamic extension of the search path. Multiple .pth files are supported, offering flexibility in configuring the Python environment.

• ._pth files: introduced to offer a more controlled environment, particularly for Python's embedded distribution.
  A ._pth file specifies a fixed set of directories that Python will include in its module search path, overriding the default behavior, including the processing of .pth files. That makes sure the embedded Python interpreter operates in a predictable and isolated manner, using only the specified directories. Only one ._pth file is recognized and processed by the interpreter, making sure a singular configuration for the environment.

That means that:

• without a ._pth file, Python populates its search path based on a series of fallbacks including the current directory, PYTHONPATH environment variable, application paths specified in the registry, and paths relative to the Python home. That process makes sure Python can locate its libraries and modules in various configurations.

• for applications that bundle Python (especially embedded distributions), a ._pth file provides a means to precisely control the search path, ignoring other configurations such as PYTHONPATH, registry entries, and even the site module unless explicitly included. That feature is important for maintaining application integrity and avoiding conflicts with other Python installations.

You do get from the documentation clear guidelines for distributing Python within an application, including the use of ._pth files, managing environment variables, and making sure of the presence of landmark files or directories. These practices help to create a stable and isolated Python runtime environment, which is particularly important for embedded applications or when Python is bundled with another product.

In short: While .pth files offer flexibility in extending Python's module search path in standard environments, ._pth files provide a mechanism for creating a controlled, isolated environment for embedded Python distributions.

---

The "Using Python on Windows / The embeddable package" documentatin focuses on the embeddable package, suggesting that ._pth files are primarily relevant in this context for specifying module search paths in a tightly controlled manner.

The use cases:

• standard distributions typically rely on .pth files for extending the module search paths,
• while embedded distributions can utilize ._pth files for isolation.

python/cpython issue 86418 mentions "Overriding the ._pth file should just be a matter of replacing the file", so again, I thought there was only one.

The documentation "The initialization of the sys.path module search path" states that the presence of a ._pth file completely overrides the normal initialization of sys.path, including ignoring all registry and environment variables. That, again, look that only one ._pth file is recognized and processed, as its existence dictates a unique, controlled environment configuration for the Python interpreter.

"Note that .pth files (without leading underscore) will be processed normally by the site module when import site has been specified."

That distinction clarifies that .pth and ._pth files are treated differently, with .pth files being processed in the standard manner by the site module, while ._pth files, when present, enforce an isolated environment that disregards other conventional mechanisms for extending sys.path.