Python-SocketIO , Yoloface: Is there a possibility that a function call is blocking my main thread?

35 Views Asked by At

I am working on a Flask Server with SocketIO. I emplemented an event handler and this is working fine (i implemented the client in a Adroid Studio Project). I can emit and receive Messages without any Problem.

I also want to use Yoloface for Facedetection and i am calling the function with scan_face(). I encountered a problem that scan_face() will only return the correct value for the first time i receive a scan_request from the Client. Everytime there is another scan_request from the client the function is not even called and if i try to use breakpoints to debug the breakpoints are skipped, scan_face() not called and the server continues to listen to the client

this is my output from the console:

flask run --host='0.0.0.0'
----- info -----
[i] Source of the camera:  0
[i] Path to output directory:  outputs/
###########################################################

==> Skipping create the outputs/ directory...
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://x.x.x.x:5000
 * Running on http://x.x.x.x:5000
Press CTRL+C to quit
x.x.x.x - - [25/Mar/2024 23:29:04] "GET /socket.io/?EIO=4&transport=polling HTTP/1.1" 200 -
Client connected to default namespace
x.x.x.x - - [25/Mar/2024 23:29:04] "POST /socket.io/?EIO=4&transport=polling&sid=iiua90AsbvoVBspJAAAA HTTP/1.1" 200 -
x.x.x.x - - [25/Mar/2024 23:29:04] "GET /socket.io/?EIO=4&transport=polling&sid=iiua90AsbvoVBspJAAAA HTTP/1.1" 200 -
x.x.x.x - - [25/Mar/2024 23:29:04] "GET /socket.io/?EIO=4&transport=polling&sid=iiua90AsbvoVBspJAAAA HTTP/1.1" 200 -
x.x.x.x - - [25/Mar/2024 23:29:04] "POST /socket.io/?EIO=4&transport=polling&sid=iiua90AsbvoVBspJAAAA HTTP/1.1" 200 -
check1
Received scan request: Scanne Person
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
200
227
254
[i] ==> # detected faces: 1
############################################################
value1
2gesichter
check2
person sent
2
check1
Received scan request: Scanne Person
check1
Received scan request: Scanne Person

This is where i call the scan_face() function:

@socketio.on('scan_request')
    def handle_scan_request(message):
        print('check1')
        print(f"Received scan request: {message}")
        scan_result = scan_face() # here i call the function which works only for the first time
        print('check2')
        if scan_result == 1:
            get_person_by_id()
        elif scan_result == 0:
            person_not_detected()

This is the whole code of my Eventhandler:

from flask import Flask, jsonify
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from flask_socketio import SocketIO, leave_room, join_room, emit
from flask import request
from sqlalchemy import func
from flask_socketio import send
from yoloface import scan_face
from PyQt5.QtCore import QTimer, QEventLoop, QCoreApplication, QThread, pyqtSignal

db = SQLAlchemy()
migrate = Migrate()
socketio = SocketIO(cors_allowed_origins="*")
last_sent_person_id = 1


def create_app():
    global last_sent_person_id

    app = Flask(__name__)
    app.config['DEBUG'] = True
    app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.db'
    db.init_app(app)
    migrate.init_app(app, db)
    socketio.init_app(app)

    from models import Person

    @app.route('/', methods=['GET', 'POST'])
    def handle_request():
        return 'Successful Connection'

    @app.route('/api/persons', methods=['GET'])
    def get_all_persons():
        persons = Person.query.all()
        result = []
        for person in persons:
            result.append({
                'id': person.id,
                'first_name': person.first_name,
                'last_name': person.last_name,
                'last_seen': person.last_seen,
                'status': person.status,
                'notes': person.notes,
                'pronouns': person.pronouns,
                'appointment': person.appointment,
                'notRecognized': person.notRecognized,
                'notSeenInReminderPeriod': person.notSeenInReminderPeriod,
                'image': person.image
            })
        return jsonify(result)

    def delete_all_data():
        with app.app_context():
            db.session.query(Person).delete()
            db.session.commit()

    @app.route('/delete_data', methods=['GET', 'POST'])
    def delete_data():
        if request.method == 'POST':
            delete_all_data()
            return 'All data deleted successfully'
        else:
            return 'This endpoint accepts POST requests only'

    @app.route('/create_dummys', methods=['POST', 'GET'])
    def create_dummys():
        if request.method == 'POST':
            from dummy_data import create_dummy_data
            create_dummy_data()
            return 'Dummy data created successfully'
        elif request.method == 'GET':
            return 'This endpoint accepts POST requests only'

    @socketio.on('get_person_by_id')
    def get_person_by_id():
        global last_sent_person_id
        person = Person.query.filter_by(id=last_sent_person_id).first()

        if person:
            print("person sent")
            emit('person_data', {
                'id': person.id,
                'first_name': person.first_name,
                'last_name': person.last_name,
                'last_seen': person.last_seen,
                'status': person.status,
                'notes': person.notes,
                'pronouns': person.pronouns,
                'appointment': person.appointment,
                'notRecognized': 'False',
                'notSeenInReminderPeriod': person.notSeenInReminderPeriod,
                'image': person.image
            })

        else:
            emit('server_message', 'Person with ID 1 not found!', broadcast=True)

        last_sent_person_id += 1
        print(last_sent_person_id)

    @socketio.on('send_empty_person')
    def person_not_detected():
        emit('person_not_detected', {
            'id': 0,
            'first_name': 0,
            'last_name': 0,
            'last_seen': 0,
            'status': 0,
            'notes': 0,
            'pronouns': 0,
            'appointment': 0,
            'notRecognized': 'False',
            'notSeenInReminderPeriod': 'True',
            'image': 0
        })

    @socketio.on('connect')
    def handle_connect():
        print('Client connected to default namespace')
        emit('server_message', 'Willkommen beim WebSocket-Server!', broadcast=True)

    @socketio.on('disconnect')
    def handle_disconnect():
        print('Client disconnected from default namespace')

    @socketio.on('message')
    def handle_message(data):
        print('received message: ' + data)
        send(data, broadcast=True)

    @socketio.on('scan_request')
    def handle_scan_request(message):
        print('check1')
        print(f"Received scan request: {message}")
        scan_result = scan_face()
        print('check2')
        if scan_result == 1:
            get_person_by_id()
        elif scan_result == 0:
            person_not_detected()

    from namespace import PersonNamespace
    socketio.on_namespace(PersonNamespace('/persons'))

    return app

And this is the yoloface.py where de scan_face() function is defined (i changed the function from yoloface.py so that i can call it without the need of arguments)

import argparse
import asyncio
import sys
import os

from utils import *

SRC = 0
OUTPUT_DIR = 'outputs/'
MODEL_CFG = './cfg/yolov3-face.cfg'
MODEL_WEIGHTS = './model-weights/yolov3-wider_16000.weights'

#####################################################################
# print the arguments
print('----- info -----')
print('[i] Source of the camera: ', SRC)
print('[i] Path to output directory: ', OUTPUT_DIR)
print('###########################################################\n')

# check outputs directory
if not os.path.exists(OUTPUT_DIR):
    print('==> Creating the {} directory...'.format(OUTPUT_DIR))
    os.makedirs(OUTPUT_DIR)
else:
    print('==> Skipping create the {} directory...'.format(OUTPUT_DIR))

net = cv2.dnn.readNetFromDarknet(MODEL_CFG, MODEL_WEIGHTS)
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_OPENCV)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)


def scan_face():
    wind_name = 'face detection using YOLOv3'
    cv2.namedWindow(wind_name, cv2.WINDOW_NORMAL)

    output_file = ''

    cap = cv2.VideoCapture(SRC)

    value = None

    while True:

        has_frame, frame = cap.read()

        # Create a 4D blob from a frame.
        blob = cv2.dnn.blobFromImage(frame, 1 / 255, (IMG_WIDTH, IMG_HEIGHT),
                                     [0, 0, 0], 1, crop=False)

        # Sets the input to the network
        net.setInput(blob)

        # Runs the forward pass to get output of the output layers
        outs = net.forward(get_outputs_names(net))

        # Remove the bounding boxes with low confidence
        faces = post_process(frame, outs, CONF_THRESHOLD, NMS_THRESHOLD)
        print('[i] ==> # detected faces: {}'.format(len(faces)))
        print('#' * 60)

        # initialize the set of information we'll displaying on the frame
        info = [
            ('number of faces detected', '{}'.format(len(faces)))
        ]

        for (i, (txt, val)) in enumerate(info):
            text = '{}: {}'.format(txt, val)
            cv2.putText(frame, text, (10, (i * 20) + 20),
                        cv2.FONT_HERSHEY_SIMPLEX, 0.7, COLOR_RED, 2)


        cv2.imshow(wind_name, frame)

        value = len(faces)
        print("value" + str(value))

        if value == 0:
            print('0gesicht')
            break

        elif value == 0:
            print('1gesicht')
            break
        else:
            print('2gesichter')
            break

    cap.release()
    cv2.destroyAllWindows()

    if value == 0:
        return 0
    else:
        return 1

I tried a few things including multithreading but none of it works so that i can call scan_face() everytime i receive a scan request from the client.

I am out of ideas how i am able to call scan_face() consistently. Maybe i am missing something. value = len(faces)

I thought that my flask Server is running on my main thread and that the call of the scan_face() function is blocking it but it still didn't work.

Also i thought that scan_face() may not be terminated in the right way so i tried to force the termination of scan_face() with sys.exit() but it still only worked the first time i called the function.

0

There are 0 best solutions below