Can pydoc operate on a pyd, or is there an equivalent to pydoc that operates on a pyd

Question

It seems to me that pydoc only works on py source code files.

Can it be used on a compiled pyd file in some way? The error I get in attempting that is

ValueError: source code string cannot contain null bytes

which arises, obviously, due to the fact that the compiled pyd contains NUL bytes.

Answer 1

Looking at the pydoc library, specially its source code (3.9), the function importfile works with .py and .pyc, but does not with .pyo nor .pyd files.

It may be possible to write custom code to parse the content of a .pyd file and re-use parts of pydoc to generate the documentation, but I think it is not simple.

Another way is through the builtin help that calls pydoc. here is a proof of concept :

# file: main.py
"""
Hello pydoc !
"""

# file: setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

ext_modules = [
    Extension("main",  ["main.py"]),
]

setup(
    name='My pyd file',
    cmdclass={'build_ext': build_ext},
    ext_modules=ext_modules
)

(I have installed Cython in my venv, so that I can compile .py files into .pyd)

I run python setup.py build_ext --inplace which produces the main.cp36-win_amd64.pyd file.
I move this file into another directory (to not mess the import later), and I use this file :

# file: get_pydoc.py
import main

if __name__ == "__main__":
    help(main)

which is in the same directory than the .pyd file (and nothing else). When ran, it prints :

Help on module main:

NAME
    main - Hello pydoc !

DATA
    __test__ = {}

FILE
    c:\\pycharmprojects\stack_overflow\q68002833_check\main.cp36-win_amd64.pyd

which proves it is possible.

Looking at the CPython implementation for help as of 3.9 we can see it only does return pydoc.help(*args, **kwds). So we can call pydoc directly :

# file: get_pydoc.py
import pydoc

import main

if __name__ == "__main__":
    pydoc.help(main)

and still have the same output.

But it prints its output directly on stdout, which is not convenient. Looking at its Cpython 3.9 implementation that it uses by default its output field, which is a property (see here) that returns sys.stdout if no output has been provided to the constructor, which is the case for the default Helper.

So by constructing a new Helper object whose output is an in-memory io.StringIO, we have the following code :

# file: get_pydoc.py
import io
import pydoc

import main

if __name__ == "__main__":
    doc_content = io.StringIO()
    helper = pydoc.Helper(output=doc_content)
    helper.help(main)
    print(doc_content.getvalue())

I think it answers your question : you can write a script that will get the pydoc from a .pyd file. It is just not as straightforward, because it is very less common.