Threads, the Python GIL and save/restore thread state

37 Views Asked by At

I'm working on making Python threads cooperate with SWI-Prolog threads, where SWI-Prolog supports fully independent native threads. We are talking about embedding Python. After initializing the embedded Python interpreter, the calling thread has a Python thread state. To allow other threads, I need to call PyEval_SaveThread() and before working with Python again from this thread I need to call PyEval_RestoreThread(). So, I think the logic is

  • Thread M initialized Python.
  • Thread M calls tstate = PyEval_SaveThread()
  • If I want to make a call to Python
    • If I am thread M
      • If tstate != NULL, call PyEval_RestoreThread(tstate) and set tstate to NULL else call PyGILState_Ensure() (recursive call)
      • Increment nesting
      • call Python
      • Decrement nesting
      • If nesting == 0, save the state again using tstate = PyEval_SaveThread() else call PyGILState_Release()
    • else
      • call PyGILState_Ensure()
      • call Python
      • call PyGILState_Release()

While this seems to work, the asymmetry is rather inconvenient and feels fragile. Is there some way to get in a symmetric situation such that I can simply get/release the GIL without bothering about the calling thread or thread state?

0

There are 0 best solutions below