How Event Driven Architecture Know When Request is Complete?

Question

Let's say there's a service called "CreateOrder". Let's say as well that the service depends on "Inventory" service to check whether the items in the order are available, and also "Location" service that determines whether the shipping destination is within the supported area (not necessarily an efficient design but put that aside for now).

In the classical approach it is easy. The client just call the "CreateOrder" service, then CreateOrder will call Inventory and Location services (either sequentially or parallel). Only then when both services returns "OK" that CreateOrder return successfully to the client with an OrderId, for example.

But how about the event driven approach? In my understanding an event driven approach works like this (please correct me if I'm wrong):

1. The client calls the CreateOrder service
2. Create order post the request into a message broker (let's say kafka) into a pre-defined topic (let's say "Inventory" and "Location" topics)
3. The Inventory and Location service consume the event from the message broker and process the request

But then, what? How the CreateOrder service get the result so it can get back to the client? One approach that come to mind is that the Inventory and Location services can write to somewhere upon completion (maybe a database), which the CreateOrder service can regularly poll for a result based on some kind of identifier (presumably a request ID). But this sounds horribly inefficient to me.

Is there a better approach? Another idea that come to mind is that the Inventory and Location service can push an event to kafka again upon completion, which the CreateOrder service can subscribe to. But there are a lot of questions with this approach: there are pontentially many requests that are running in parallel, how can the CreateOrder service selectively get the event for the appropriate request? To make it more complicated, what if the "CreateOrder" service is deployed in multiple instances in multiple machine? If this is even possible, is it efficient to do so? And is this possible at all? How to make the RestController wait for the kafka event in the first place?

Please enlighten me. A sample code (preferably in Spring-Boot) is always welcome!

Thanks.

Answer 1

One nice benefit of asynchronous event-driven systems is that they make the latency inherent in disseminating information that is fundamental to one's real problem apparent. Since this latency is a fundamental part of the real world (the speed of light exists, for starters), this means that there's an easy way to think of a solution: figure out how you'd solve it with people communicating and you can then transfer that solution to code and basically treat computers running software as people who can process paperwork really quickly and communicate somewhat reliably.

So imagine you're what used to be called a mail-order or direct-sales merchant. People call you up saying what they'd like to buy.

What would the process for such a phone call be? Are you going to put the caller on hold (better have some really good on-hold music...) while you call the warehouse to validate that what they want is actually in stock?

Chances are, you'll write the order down without putting the caller on hold. You give them an order number that they can use to inquire about the order, and you've probably also obtained some means by which you can contact them if there's some issue with the order. Once you've obtained that information, you say "thank you, your estimated ship time is... and you should get them delivered by..." and you can hang up. Then you forward the order to the warehouse for fulfillment.

Note that, unless you keep the caller on the phone until the package is delivered, you're going to have to provide them with the order number and/or be able to contact them later if there's some problem (that's a pretty boring call, though you're kind of like a sportscaster, which is fun, at least the first time: "and the warehouse worker has your item, and they tripped..."). The data returned by the inventory service is out of date by the time the order creation service receives it.

So your order creation service considers the order created when it's been durably published (to Kafka in this case). 201 Accepted is, as OneCricketeer notes, a perfect HTTP code for that. The other services will publish notable events about the state of the order (e.g. out of stocks, ready for shipment, payment reserved, etc.) for the consumption of a service which allows the requestor (or their designee) to query for a view (hopefully reasonably current) of the order. There might even be services watching those events in order to send proactive communications.

Of course, the obvious question is what if you want to be able to alert the requestor that some items are out of stock. Keeping in mind that in any request-response interaction, by the time the requestor receives the response it's stale (in our phone example, the warehouse could say that they have six widgets, hang up, and then a forklift goes haywire and drives into a shelving unit... oopsie), this implies that the process of order creation is, well, a process. Creating an order isn't a one-shot request-response, it's a dialogue, each request working an in-progress order that can then be finalized. This is where CQRS can come in handy: your order creation service maintains a view of enough of the inventory state to fulfill its duties. In our phone example, this could be a post-it note on the desk with notable out-of-stocks from the warehouse.

Answer 2

The Inventory and Location services can also have their own topics for the purpose of streaming/logging their responses. Your order service can subscribe to these topics and filter on keys (i.e. OrderId) that they care about. Since your order thread is waiting for a pertinent event from both topics, you will need to consider timeout scenarios/error handling.