flask_socketio sometimes fail to emit after deployed on Heroku

Question

I am new to app development. I am using Flask and Flask_socketio for the server and socket.io-client for the client. When I trigger getSimData from client side, the server fails to emit file_ready data back, but when I refresh the web page and trigger getSimData again, sometimes the emit would work.

When emit failed, the log would return

2023-09-26T17:48:36.771176+00:00 app[web.1]: emitting event "file_ready" to bPUY63vpR_BRYu0tAAAH [/progress]
2023-09-26T17:48:36.771192+00:00 app[web.1]: Sid bPUY63vpR_BRYu0tAAAH, File emit done

When emit is successfull, the log would return

2023-09-26T17:51:58.588447+00:00 app[web.1]: emitting event "file_ready" to HYM2BOFXCPoJ51UKAAAF [/progress]
2023-09-26T17:51:58.588574+00:00 app[web.1]: EvYgo_hrowx0Ft6uAAAE: Sending packet MESSAGE data 2/progress,1["file_ready",{"download_url":"downloadable_link","sid":"HYM2BOFXCPoJ51UKAAAF"}]
2023-09-26T17:51:58.588589+00:00 app[web.1]: Sid HYM2BOFXCPoJ51UKAAAF, File emit done
2023-09-26T17:51:58.683620+00:00 app[web.1]: EvYgo_hrowx0Ft6uAAAE: Received packet MESSAGE data 3/progress,1[null,"Acknowledged"]
2023-09-26T17:51:58.683687+00:00 app[web.1]: received ack from HYM2BOFXCPoJ51UKAAAF [/progress]
2023-09-26T17:51:58.683710+00:00 app[web.1]: Emit successful: Acknowledged

On the server side, I am passing sid and emit wrapper to my main compute function which will be executed on the background by a thread.

Here is the app.py

app = Flask(__name__)
origins = [
    "https://my-solution.app"
]
CORS(app, resources={r"/*": {"origins": origins}})
# heroku features:enable http-session-affinity --app satellite-solver
socketio = SocketIO(app, cors_allowed_origins=origins, logger=True, engineio_logger=True)

class MyWorker():
    def __init__(self, current_datetime, sid, params):
        self.sid = sid
        self.params = params

        socketio.start_background_task(target=self.run)

    def run(self):
        print(f'Running MyWorker for SID {self.sid}')

        def emit_wrapper(*args, **kwargs):
            return socketio.emit(*args, **kwargs)

        compute(sid=self.sid, socket_emit=emit_wrapper, **self.params)

@socketio.on('connect', namespace='/progress')
def handle_connect():
    print("Client connected to /progress namespace.")

@app.route('/getSimData', methods=['POST'])
def get_sim_data():
    sid = request.form.get("sid")
    ...
    # Start the background task
    MyWorker(sid, params)

    return jsonify({"message": f"{sid} Processing started!"})

if __name__ == '__main__':
    socketio.run(app, threaded=True, port=5000)

Here is the compute function and emit code

def emit_callback(error=None, response=None):
    global emit_success
    if error:
        print("Emit failed:", error)
        emit_success = False
    else:
        print("Emit successful:", response)
        emit_success = True

def compute(sid=None, socket_emit=None, **kwargs):
        ...
        socket_emit('file_ready', {'download_url': url, 'sid': sid},
                    namespace='/progress', room=sid, callback=emit_callback)
        print(f'Sid {sid}, File emit done')

On the client side, I'm listening for the file_ready event.

        // Connect to the server's socket
        const socket = io.connect('https://myapp.herokuapp.com/progress', {
            transports: ['websocket'],
            reconnection: true,
            reconnectionAttempts: 10,
            reconnectionDelay: 1000
        });

        socket.on('connect', () => {
            console.log("Establishing connection with server...");
            console.log("Connected with SID:", socket.id);
            data.append('sid', socket.id);

            fetch(baseUrl, {
                method: 'POST',
                body: data})
                .then(response => response.json())
                .then(data => {
                    console.log(data.message);
                })
                .catch(error => console.error('Error:', error));
        });

        // Listen for the 'file_ready' event
        socket.on('file_ready', function(data, ack) {
            console.log(`File Ready is ${(data.sid === socket.id)}: \n data.sid ${data.sid} \n socket.id ${socket.id}`);
            if (data.sid === socket.id) {
                ack(null, "Acknowledged");
                console.log("File ready:", data.download_url);
                const downloadURL = data.download_url;

                // Fetch the file from the download URL
                fetch(downloadURL)
                    .then(response => response.blob())  // Convert the fetched data to a blob
                    .then(blob => {
                        const blobUrl = window.URL.createObjectURL(blob);

                        const downloadAnchor = document.createElement('a');
                        downloadAnchor.href = blobUrl;
                        downloadAnchor.download = "output.zip";
                        document.body.appendChild(downloadAnchor);
                        downloadAnchor.click();
                        document.body.removeChild(downloadAnchor);
                        window.URL.revokeObjectURL(blobUrl);

                        socket.disconnect();
                        fadeOutElement(progressContainer);
                        fadeOutElement(loadingText);
                    })
                    .catch(error => {
                        console.error("Error while fetching and downloading file:", error);
                        socket.disconnect();
                        fadeOutElement(progressContainer);
                        fadeOutElement(loadingText);
                    });
            }
        });

Would really appreciate some guidance on how to fix this issue.

I tried adding retries and callback, but it didn't help. It looks like from the log, when emit failed it didn't send out data to client. However the emit and print function seem triggered correctly.

Here is my requirements.txt

Flask==2.3.3
Flask_Cors==4.0.0
Flask_SocketIO==5.3.6
Flask_Executor==1.0.0
numpy==1.25.2
pandas==2.1.0
gunicorn==21.2.0
eventlet==0.33.3
boto3==1.28.53

and Procfile

web: gunicorn app:app --worker-class eventlet