C# Async-Await throwing AggregateException on Task.WaitAll

Question

I have a Quartz.NET recurring job based on cron expression in a .NET 8 Worker Service. The job is supposed to invoke 2 external APIs and then perform some processing on the API responses downstream.

To increase throughput, I have invoked the APIs using Task Parallel Library. So, I have to wait for both the tasks to finish before I can start processing. The implementation looks somewhat like this:

private void GetCarStatusAndPositionalInformation(
    IJobExecutionContext context, 
    string terminalNumber)
{
    CarInfo? carInfo = null; CarStatusInfo? carStatusInfo = null; 
    try
    {
        var getCarInformation = Task.Run(async () =>
        {
            var carInfoResponse = await _apiClient.QueryCarInformation(
                terminalNumber, context.CancellationToken);

            if (carInfoResponse?.CarInfo != null)
            {
                carInfo = carInfoResponse.CarInfo;
            }
            await Task.CompletedTask;
        },
        context.CancellationToken)
        .ContinueWith(
            continuationAction =>
            {
                _logger.LogWarning("Failed to get car info for terminal number: {terminalNumber}", terminalNumber);
            },
            context.CancellationToken,
            TaskContinuationOptions.OnlyOnFaulted,
            TaskScheduler.Current);

        var getCarStatusInformation = Task.Run(async () =>
        {
            var carStatusInfoResponse = await _apiClient.GetCarStatusInfoAsync(
                terminalNumber, context.CancellationToken);

            if (carStatusInfoResponse?.CarStatusInfo != null)
            {
                carStatusInfo = carStatusInfoResponse.CarStatusInfo;
            }
            await Task.CompletedTask;
        },
        context.CancellationToken)
        .ContinueWith(
            continuationAction =>
            {
                _logger.LogWarning("Failed to get car info for terminal number: {terminalNumber}", terminalNumber);
            },
            context.CancellationToken,
            TaskContinuationOptions.OnlyOnFaulted,
            TaskScheduler.Current);

        

        Task.WaitAll(
            [getCarInformation, getCarStatusInformation], timeout: TimeSpan.FromSeconds(30));

        if (carInfo != null) && carStatusInfo != null) 
        { 
            // Do some processing
        }
    }
    catch (AggregateException aex)
    {

    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "Failed to obtain car status & positional information for: {terminalNumber}", terminalNumber);
    }
}

Why am I always getting AggregateException with InnerException saying the tasks have been canceled?

Answer 1

The problem lies within the way you constructed the Tasks.

getCarInformation and getCarStatusInformation actually reference the continuations, not the Tasks you were expecting.

Now, the "fast and dirty" solution would be to take two steps:

var getCarInformation = Task.Run(async () =>
        {
            var carInfoResponse = await _apiClient.QueryCarInformation(
                terminalNumber, context.CancellationToken);

            if (carInfoResponse?.CarInfo != null)
            {
                carInfo = carInfoResponse.CarInfo;
            }
            await Task.CompletedTask; // <- this makes no sense, btw
        },
        context.CancellationToken);
// ^^ Now you are referencing the Task you expected to reference
getCarInformation 
        .ContinueWith(
            continuationAction =>
            {
                _logger.LogWarning("Failed to get car info for terminal number: {terminalNumber}", terminalNumber);
            },
            context.CancellationToken,
            TaskContinuationOptions.OnlyOnFaulted,
            TaskScheduler.Current);

But cleaner would be to not use ContinueWith at all. Just move the failure-logging into the Task itself.

var getCarInformation = Task.Run(async () =>
{
    try
    {
            var carInfoResponse = await _apiClient.QueryCarInformation(
                terminalNumber, context.CancellationToken);

            if (carInfoResponse?.CarInfo != null)
            {
                carInfo = carInfoResponse.CarInfo;
            }
    } 
    catch (Exception)
    {
        _logger.LogWarning("Failed to get car info for terminal number: {terminalNumber}", terminalNumber);
        throw; // Or better not throw? Depends on what you need to do.
    }
}, context.CancellationToken);

Or just let the Exceptions bubble up. But be aware that this has some caveats. Especially if you need to handle both (if both fail). In that WhenAll (and I am not sure about WaitAll) would then swallow exceptions.

See: https://github.com/dotnet/core/issues/7011#issuecomment-988018459

Answer 2

Write two ordinary async methods with exception handlers. Example:

private async Task GetCarInformation() 
{
    try
    {
        var carInfoResponse = await _apiClient.QueryCarInformation(
            terminalNumber, context.CancellationToken);

        if (carInfoResponse?.CarInfo != null)
        {
            carInfo = carInfoResponse.CarInfo;
        }
    }
    catch(HttpRequestException exception)
    {
        _logger.LogWarning("...");
    }
}

Now call them and wait for them.

Task.WaitAll( new[] {
    Task.Run(GetCarInformation),
    Task.Run(GetCarStatusInformation)
    },
    timeout: TimeSpan.FromSeconds(30)
);

Or if you want to be stingy with threads:

Task.Run( async () =>
    {
        await Task.WhenAll( new [] {
            GetCarInformation(),
            GetCarStatusInformation()
            });
    },
    timeout: TimeSpan.FromSeconds(30)
).Wait();