Preventing a WCF client from issuing too many requests

Question

I am writing an application where the Client issues commands to a web service (CQRS)

• The client is written in C#
• The client uses a WCF Proxy to send the messages
• The client uses the async pattern to call the web service
• The client can issue multiple requests at once.

My problem is that sometimes the client simply issues too many requests and the service starts returning that it is too busy.

Here is an example. I am registering orders and they can be from a handful up to a few 1000s.

var taskList = Orders.Select(order => _cmdSvc.ExecuteAsync(order))
                     .ToList();

await Task.WhenAll(taskList);

Basically, I call ExecuteAsync for every order and get a Task back. Then I just await for them all to complete.

I don't really want to fix this server-side because no matter how much I tune it, the client could still kill it by sending for example 10,000 requests.

So my question is. Can I configure the WCF Client in any way so that it simply takes all the requests and sends the maximum of say 20, once one completes it automatically dispatches the next, etc? Or is the Task I get back linked to the actual HTTP request and can therefore not return until the request has actually been dispatched?

If this is the case and WCF Client simply cannot do this form me, I have the idea of decorating the WCF Client with a class that queues commands, returns a Task (using TaskCompletionSource) and then makes sure that there are no more than say 20 requests active at a time. I know this will work but I would like to ask if anyone knows of a library or a class that does something like this?

This is kind of like Throttling but I don't want to do exactly that because I don't want to limit how many requests I can send in a given period of time but rather how many active requests can exist at any given time.

Answer 1

Based on @PanagiotisKanavos suggjestion, here is how I solved this.

RequestLimitCommandService acts as a decorator for the actual service which is passed in to the constructor as innerSvc. Once someone calls ExecuteAsync a completion source is created which along with the command is posted to the ActonBlock, the caller then gets back the a Task from the completion source.

The ActionBlock will then call the processing method. This method sends the command to the web service. Depending on what happens, this method will use the completion source to either notify the original sender that a command was processed successfully or attach the exception that occurred to the source.

public class RequestLimitCommandService : IAsyncCommandService
{
    private class ExecutionToken
    {
        public TaskCompletionSource<bool> Source { get; }
        public ICommand Command { get; }

        public ExecutionToken(TaskCompletionSource<bool> source, ICommand command)
        {
            Source = source;
            Command = command;
        }
    }

    private IAsyncCommandService _innerSrc;
    private ActionBlock<ExecutionToken> _block;

    public RequestLimitCommandService(IAsyncCommandService innerSvc, int maxDegreeOfParallelism)
    {
        _innerSrc = innerSvc;
        var options = new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = maxDegreeOfParallelism };
        _block = new ActionBlock<ExecutionToken>(Execute, options);
    }

    public Task IAsyncCommandService.ExecuteAsync(ICommand command)
    {
        var source = new TaskCompletionSource<bool>();
        var token = new ExecutionToken(source, command);
        _block.Post(token);
        return source.Task;
    }

    private async Task Execute(ExecutionToken token)
    {
        try
        {
            await _innerSrc.ExecuteAsync(token.Command);
            token.Source.SetResult(true);
        }
        catch (Exception ex)
        {
            token.Source.SetException(ex);
        }
    }   
}