How can I publish/subscribe to messages without any loss?

104 Views Asked by At

I've been developing an async Rust library for the MAVLink Protocol that will communicate with UAVs.

The library follows a communication flow as described: diagram of the flow

The message publisher task receives incoming bytes from the underlying connection (usually a UDP socket), and parse the bytes into messages, and then publish them to other tasks, for example the heartbeat task, or the command protocol task. Currently I use tokio::sync::watch channels for publishing messages. I know this is a bad decision, and I'm open to any suggestions on that issue too.

Each task usually contain a loop like the one below:

loop {
    let message = wait_for(message_id).await;
    process_message(message)
}

As you can see, tasks receive messages from the publisher by calling wait_for(). And then they process the received message.

Now that you have some idea how the library works, I can start explaining my problem.

My problem with this flow is that when tasks process_message, they become unable to wait_for incoming messages. This causes tasks to miss some messsages, which is the main problem I'm struggling with.

Possible Solutions:

  • I could register some callbacks and solve this problem, but I want to deliver a future based API to the end user. Maybe I could use callbacks internally?.

  • I could create a ring buffer for each task, and hold messages to some extent (for example 32 messages at max.), and allow messages to be lost, but would that be overcomplicating things?

Everything aside, my design might be flawed, so I'm open to new designs ideas too. In this case, how can I design my library so that tasks never or almost never miss messages?

Finally, is there a "pattern" that would solve this problem? Or any kind of resource that might inspire me

0

There are 0 best solutions below