Performing bidirectional signals in QThreads

Question

I've been wondering over the question: is there are correct (or standard) way to create QThreads, which make of use of bidirectional signals?

That is, QThreads capable of processing not only custom workers in another thread, but also capable of processing events and signals incoming from the main thread, and also, capable of emitting signals to the main thread, so both threads can exchange data between each other using a single model: qt signals.

Most examples I've found so far (in articles, documentation, even here on SO) make use of unidirectional signals only, emitting them from the QThread to the main thread, to safely call non thread-safe QObject methods (that should be called only in the main thread).

The reason I would like to know if there's a such way to use bidirectional connection, is for the safety of passing data from the main thread to the QThread, without the need of explicitely using QMutexes or similar kinds of locks in order to exchange data, which is something that the Qt5 documentation doesn't cover, as far as I know.

I'm not saying that using bidirectional signals will automatically solve the problem for thread-safety in all forms. Reading/writting in shared memory or files, for example, would still require the usage of threading locks. I would just like to know if there is some kind of documentation about this question, or if anyone here has invented a working way to perform it.

Answer 1

Thread-safe bidirectional signals are quite easy to implement without any special handling. In most cases, Qt will automatically detect cross-thread signals and post an event with the serialised signal parameters (if any) to the event-queue of the receiving thread. This mechanism is the main reason for using moveToThread rather than a QThread sub-class: as explained in the Qt docs, "the default implementation [of QThread.run] simply calls exec()", which starts the local event-loop. So, unless your sub-class explicitly calls exec, it won't process any of the events it might receive (including signal and timer events). Of course, there are many legimate use-cases where this doesn't matter (as you can appreciate from reading the article you already linked to) - so it's simply a matter of choosing the right Qt API for the job. In addition to thread-safe bidirerctional signals, Qt also provides an API for more directly invoking methods using the same underlying mechanism: QMetaObject.invokeMethod. Taken together, these should be more than adequate for most situations.

Below is a basic example that illustrates some of the main techniques. Hopefully the code will be mostly self-explanatory, since it's mainly a matter of connecting/invoking the signals and slots in the right way. But note that calling QCoreApplication.processEvents is required within the while-loop, since it would otherwise block the worker's event-loop processing. The debugging output shows which thread each slot is being invoked in:

import threading
from PyQt5 import QtCore, QtWidgets

def check_thread(text):
    print(f'{text}: {threading.current_thread().name}')

class Worker(QtCore.QObject):
    countChanged = QtCore.pyqtSignal(int)

    def __init__(self):
        super().__init__()
        self._active = False
        self._step = 1

    def process(self):
        check_thread('process')
        count = 0
        self._active = True
        while self._active:
            QtCore.QThread.msleep(500)
            QtCore.QCoreApplication.processEvents()
            count += self._step
            self.countChanged.emit(count)
        self._active = False

    @QtCore.pyqtSlot()
    def stop(self):
        check_thread('stopping')
        self._active = False

    @QtCore.pyqtSlot(int)
    def handleStepChanged(self, step):
        check_thread('step-change')
        self._step = step

class Window(QtWidgets.QWidget):
    def __init__(self):
        super().__init__()
        self.buttonStart = QtWidgets.QPushButton('Start')
        self.buttonStart.clicked.connect(self.startWorker)
        self.buttonStop = QtWidgets.QPushButton('Stop')
        self.buttonStop.clicked.connect(self.stopWorker)
        self.buttonStop.setEnabled(False)
        self.labelOutput = QtWidgets.QLabel()
        self.spinbox = QtWidgets.QSpinBox()
        self.spinbox.setRange(1, 10)
        layout = QtWidgets.QGridLayout(self)
        layout.addWidget(self.labelOutput, 0, 0, 1, 3)
        layout.addWidget(QtWidgets.QLabel('Step:'), 1, 0)
        layout.addWidget(self.spinbox, 1, 1)
        layout.addWidget(self.buttonStart, 1, 2)
        layout.addWidget(self.buttonStop, 1, 3)
        self.thread = QtCore.QThread(self)
        self.worker = Worker()
        self.worker.moveToThread(self.thread)
        self.thread.started.connect(self.worker.process)
        self.spinbox.valueChanged.connect(self.worker.handleStepChanged)
        self.worker.countChanged.connect(self.handleCountChanged)
        self.handleCountChanged()

    @QtCore.pyqtSlot(int)
    def handleCountChanged(self, count=0):
        check_thread('count-change')
        self.labelOutput.setText(f'<p>Current Count: <b>{count}</b></p>')

    def startWorker(self):
        check_thread('start')
        if not self.thread.isRunning():
            self.thread.start()
            self.buttonStart.setEnabled(False)
            self.buttonStop.setEnabled(True)

    def stopWorker(self):
        check_thread('stop')
        if self.thread.isRunning():
            self.buttonStart.setEnabled(True)
            self.buttonStop.setEnabled(False)
            QtCore.QMetaObject.invokeMethod(self.worker, 'stop')
            self.thread.quit()
            self.thread.wait()

    def closeEvent(self, event):
        self.stopWorker()

if __name__ == '__main__':

    app = QtWidgets.QApplication(['Test'])
    window = Window()
    window.setGeometry(600, 100, 200, 100)
    window.show()
    app.exec()

Answer 2

After experimenting with QThreads for more or less an year, I've come to the conclusion that exists 2 forms of creating and using QThreads in Qt5 (with or without python):

1. Subclassing QThread;
2. Subclassing QObject as a worker, and calling QObject.moveToThread(QThread).

About the first one, there's a lot of discussion in many forums online, which many developers take a side: they are either ok with it, while most of them are not ok with subclassing QThread. In my own opinion, there's no problem in subclassing it, unless you know what you're doing. There's always the potential that Signals and Slots connections might fail the way you expect, as Qt will always try its best effort to detect which types of connetions that should be made (if you're using Qt.AutoConnection); And there's a possibiliy that the QThread's event loop will never be executed, if the developer forgets to call super().run() for example.

In the case of this question, this is something bad, as there would be no possibility to run bidirectional signals between the main thread and the other thread, as signals rely on an QEventLoop of some sort to be scheduled and processed.

At some articles and other materials I've found on the internet, even though they're not recent, they transmit the message that most authors want to, by encouraging the second approach, which is by creating a Worker (QObject subclass) and using QObject.moveToThread, together with one of the 2 options below:

• A connection with the Worker's method and the QThread.started signal; or,
• A Worker's slot connection with a custom signal of choice, such as a click of a button;

The first option executes all connected methods in an specific order (based on the Signal.connect sequence calls). As the QThread.started signal will be called from the spawned thread context, the signal type will probably default to a Qt.DirectConnection, blocking the event loop entirely until all connected listeners have finished executing. This means that events from the main thread to the other thread, will be ignored until the QEventLoop starts on the other thread.

The second option executes one or more listeners probably via a Qt.QueuedConnection, by scheduling the Worker's method to be called in the QThread's event loop, and be processing them in there. This type of scheduling happens after the QThread's QEventLoop had been started, which guarantees that events from the main thread to the spawned thread will be executed at some point (either before or after the listeners are scheduled to be executed).

However, the problem arises when you want to use iterations (or infinite iterations), such as While (True) loops. In order to do it, I would guess that one could use a QTimer, residing on the main thread, and periodically emit a custom signal at some interval, using the 2nd option, by scheduling Worker's methods to be executed on the other thread. This would be a viable option, but it would be required to keep at least two instances in memory away from the garbage collector: the QThread, and the QTimer.

Aside from using a QTimer, I've found out that one can also use the QThread's QAbstractEventDispatcher, in order to register QObjects as timers, and execute them at specific invervals of time. In order to make use of each QObject as timer, we would need to reimplement the virtual method: QObject.timerEvent (note: in the docs, there's nothing describing that it is bad to reimplement this method). So in theory it should be ok.

In essence, this is exactly what using a QTimer would perform: one or more QObject's methods would be called over and over within a single interval. The good side of this, is that we do not need to keep a QTimer's reference to control wether QObjects are being executed inside the QThread. Everything could be handled by the QThread itself, including terminating it (such as calling QThread.quit). Here's a working example on how to do it:

from PySide2.QtCore import QThread, QObject, Qt, Signal
from PySide2.QtWidgets import QApplication, QWidget, QVBoxLayout, QLabel, QPushButton
from datetime import datetime

class Worker(QObject):
    generate = Signal(str)

    def __init__(self):
        super().__init__()
        self.iter = 0

    def signalCatch(self):
        print('Signal Catched: ', self.thread())

    def timerEvent(self, event):
        # Demanding task
        dt = datetime.now()
        while ((datetime.now() - dt).total_seconds() < 1):
            pass

        self.iter += 1
        print('Iteration: ', self.iter, self.thread())
        self.generate.emit('Iteration: %d' % (self.iter))

def setup(qthr, worker):
    qthr.eventDispatcher().registerTimer(100, Qt.CoarseTimer, worker)

###
app = QApplication()
win = QWidget()
lbl = QLabel()
but = QPushButton('Send Signal')
vbx = QVBoxLayout()
vbx.addWidget(lbl)
vbx.addWidget(but)
win.setLayout(vbx)
win.show()

worker = Worker()
thread = QThread()

worker.moveToThread(thread)
worker.generate.connect(lbl.setText)
thread.started.connect(lambda: setup(thread, worker))
but.clicked.connect(worker.signalCatch)
thread.start()

app.exec_()
thread.quit()
thread.wait()

The only problem with these two approaches so far is that: I haven't found anything about them in forums, articles, or anything. Even on SO, there are no answers regarding this matter, as far as I could look into. I don't know if anyone here came up with this idea before me, and if you did, I would like to ask if you could share more details about it, such as: if this is a correct way to create QThreads + Workers with bidirectional signals, or maybe, if I made a mistake or a miscalculation somewhere.

Furthermore, I do not know the implications of this answer. I can only test this in Windows 11 (which is my OS), with PySide2. I don't know if this works for any other Qt5 binder, Qt in C++ or any other kind of operating system out there.