This is more of an architecture question.
I’ve got a current architecture which sits like this
Azure http function -> service bus -> azure bus function -> database -> azure scheduled function -> analytics database
Now I’m wondering if I could simplify all this using azure streaming analytics with 2 sinks.
My main concern is doing validation. If a sink errors, then all sinks are retried. What happens if I get a poison message in the stream? It’ll kill my system?
My current system will retry and then dead letter
So I guess, my first question. How do you stop poison messages entering streaming analytics if always retry is enabled?
I’m thinking of modifying the http function to validate, then place the messages into event grid. What happens if the message changes? Eg, the validation rules? I’m guessing I need to keep it backwards compatible?