`inspect.getsource` from a function defined in a string? `s="def f(): return 5"`

Question

Given a function defined inline, how do I get getsource to provide the output? - This is for a test, here's the kind of thing I'm trying:

from importlib.util import module_from_spec, spec_from_loader

_locals = module_from_spec(
    spec_from_loader("helper", loader=None, origin="str")  # loader=MemoryInspectLoader
)
exec(
    'def f(): return "foo"',
    _locals.__dict__,
)
f = getattr(_locals, "f")
setattr(f, "__loader__", MemoryInspectLoader)

With my attempt, as it looks like a linecache issue:

from importlib.abc import Loader

class MemoryInspectLoader(Loader):
    def get_code(self): raise NotImplementedError()

But the error is never raised. From getsource(f), I just get:

In [2]: import inspect
   ...: inspect.getsource(f)
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-3-1348c7a45f75> in <module>
----> 1 inspect.getsource(f)

/usr/lib/python3.8/inspect.py in getsource(object)
    983     or code object.  The source code is returned as a single string.  An
    984     OSError is raised if the source code cannot be retrieved."""
--> 985     lines, lnum = getsourcelines(object)
    986     return ''.join(lines)
    987 

/usr/lib/python3.8/inspect.py in getsourcelines(object)
    965     raised if the source code cannot be retrieved."""
    966     object = unwrap(object)
--> 967     lines, lnum = findsource(object)
    968 
    969     if istraceback(object):

/usr/lib/python3.8/inspect.py in findsource(object)
    796         lines = linecache.getlines(file)
    797     if not lines:
--> 798         raise OSError('could not get source code')
    799 
    800     if ismodule(object):

OSError: could not get source code

How do I make getsource work with an inline-defined function in Python 3.6+?

Answer 1

Here's my solution to this:

import os.path
import sys
import tempfile
from importlib.util import module_from_spec, spec_from_loader
from types import ModuleType
from typing import Any, Callable

class ShowSourceLoader:
    def __init__(self, modname: str, source: str) -> None:
        self.modname = modname
        self.source = source

    def get_source(self, modname: str) -> str:
        if modname != self.modname:
            raise ImportError(modname)
        return self.source


def make_function(s: str) -> Callable[..., Any]:
    filename = tempfile.mktemp(suffix='.py')
    modname = os.path.splitext(os.path.basename(filename))[0]
    assert modname not in sys.modules
    # our loader is a dummy one which just spits out our source
    loader = ShowSourceLoader(modname, s)
    spec = spec_from_loader(modname, loader, origin=filename)
    module = module_from_spec(spec)
    # the code must be compiled so the function's code object has a filename
    code = compile(s, mode='exec', filename=filename)
    exec(code, module.__dict__)
    # inspect.getmodule(...) requires it to be in sys.modules
    sys.modules[modname] = module
    return module.f


import inspect
func = make_function('def f(): print("hi")')
print(inspect.getsource(func))

output:

$ python3 t.py 
def f(): print("hi")

there's a few subtle, and unfortunate points to this:

1. it requires something injected into sys.modules (inspect.getsource always looks there for inspect.getmodule)
2. the __loader__ I've built is bogus, if you're doing anything else that requires a functioning __loader__ this will likely break for that
3. other oddities are documented inline

---

an aside, you're probably better to keep the original source around in some other way, rather than boomeranging through several globals (sys.modules, linecache, __loader__, etc.)

Answer 2

Not entirely sure if I got the question correctly.

But if you have the following code:

class MemoryInspectLoader(Loader):
    def get_code(self): raise NotImplementedError()

You can extract the functions body by using dill.

from dill.source import getsource

print(getsource(MemoryInspectLoader.get_code))

Which will output:

        def get_code(self): raise NotImplementedError()

Which is also demonstrated in this SO answer.

Answer 3

Monkey patch linecache.getlines in order to make inspect.getsource() to work with code coming from exec(). When you look at the error stack, it stops at findsource() in inspect.py. When you look at the code of findsource(), you'll see a hint:

# Allow filenames in form of "<something>" to pass through.
# `doctest` monkeypatches `linecache` module to enable
# inspection, so let `linecache.getlines` to be called.

And then if you look at this test function you'll see what it means. You can temporarily change one of the core Python function to serve your purpose.

Anyway, here's the solution:

import linecache
import inspect

def exec_getsource(code):
    getlines = linecache.getlines
    def monkey_patch(filename, module_globals=None):
        if filename == '<string>':
            return code.splitlines(keepends=True)
        else:
            return getlines(filename, module_globals)
    linecache.getlines = monkey_patch
    
    try:
        exec(code)
        #you can now use inspect.getsource() on the result of exec() here
        
    finally:
        linecache.getlines = getlines