(.NET Core & AzureAd) How to keep user-context in long running process for calling downstream API

Question

Intro / context

I have a .NET Core application and I know nothing about .NET Core which puts me in prime position for not finding a solution to this for me way too complex problem.

The application is built in a standard way. There is a Startup.cs where all kind of configuration for the dependency injection magic is done. There is also a configuration for a downstream API.

        services
            .AddMicrosoftIdentityWebApiAuthentication(Configuration)
            .EnableTokenAcquisitionToCallDownstreamApi()
            .AddDownstreamWebApi(
                "MyDownstreamApi",
                Configuration.GetSection("MyDownstreamApi"))
            .AddInMemoryTokenCaches()

Initial situation

I have a very long running process that is executed in one method of a controller. Namely the "handle" for a simple GET request that starts some heavy logic and at some point will return with a result. The method in the controller is waiting for this and only returns 200 AFTER that process is finished. That process involves calling a downstream API on-behalf-of the requesting user. In principle this works and worked in the past with current setup and configuration. BUT, in some cases the process takes too long and runs into the overall hardcoded timeout in .NET. OF COURSE, it is really bad practise in REST to keep a client waiting for 30seconds until you return a result.

Naive refactoring

So, I refactored this in a hacky way (just want to see it working in principle) and theoretically the code looks good to me.

• the method X in the controller starts the task A containing the logic that takes way too much time
• X also registers A in a singleton registry
• singleton registry returns a <guid> as a price back to X
• X returns now back to client with 200 and <guid>

=> Now, the client can come back to API anytime with that <guid> to request current status of the task and eventually result of the task.

For this the API now has (pseudo endpoints).

• PUT /long-running-logic (starts and returns <guid>)
• GET /long-running-logic/status/<guid> (for getting status)
• GET /long-running-logic/<guid> (for getting the result after status told you "I am done")

Problem

The way-too-much-time-logic involves calling the downstream API on-behalf-of the user at some point. Sadly that point of time is when the request was already answered and user context is gone in API (that is my interpretation of Microsoft.Identity.Client.MsalUiRequiredException , ErrorCode: user_null).

So I went back to research / documentation / ... I found long-running OBO processes. This has to be it, I think. But how in hell do I wire this together so it works? Here I am done and defeated.

Keep in mind, I have the additional point of the downstream API that isn't covered there.

I found out how to create a IConfidentialClientApplication (I added that one to Startup.cs) but the code I added doesn't really make any sense to me. It would be more than magic if that would work - so I expected it to not work and it doesn't work ouf course. There is the Microsoft.Identity.Client.MsalUiRequiredException: AADSTS65001: The user or administrator has not consented to use the application with ID ... error.

Is there somewhere a working example of such a use case?

In Node.js I would just keep the access_token when user requests for the first time somewhere and request a new one on-behalf-of at the time I need it for calling my downstream API in such a long running process... simple as that... But, here in .NET Core with all this blackbox magic configuration interface what-ever things going on I am completely lost and don't know which documentation I have to find to finally understand this .... :(

Btw. I have now the idea of just taking an approach bypassing all that .NET Core magic and just using simple HttpClient calls, doing the requesting on-behalf-of token by myself, controlling initial users access_token also by myself.

Any hints / help?

Thanks!

Answer 1

This is how I solved it

Part 1 of the puzzle - storing Bearer token to local variable

Task A gets as input also the AccessToken of the request that starts the long running background task A.

How to get that AccessToken? => for that I added an extension method for ControllerBase:

public static class ControllerBaseExtensions
{
    public static string AccessToken(this ControllerBase controller)
    {
        return controller.Request.Headers[HeaderNames.Authorization].ToString().Replace("Bearer ", "");
    }
}

Part 2 of the puzzle - passing Bearer token to services that call downstream API's

Since I have now the AccessToken, I can just pass it down (not via DI, just as parameter of involved methods...) to the place where it gets consumed in order to create a new one on-behalf-of in order to call the downstream API.

In background task A:

... do your stuff / long running things
_downstreamApi.CallPATCH(AccessToken, "<url>", <data>)

Part 3 of the puzzle - getting new token on-behalf-of user

I explictly create a new http client here so I can set the new token. Also, beforehand I request a new token explictly with "bare hands"... no shitty black box magic going on. Just straigthforward functional programming...

It looks like this:

public class DownstreamApiCaller
{
    private readonly IConfidentialClientApplication _app;
    private readonly IConfiguration _configuration;
    private readonly ILogger<DownstreamApiCaller> _logger;

    public DownstreamApiCaller(
        IConfiguration configuration,
        ILogger<DownstreamApiCaller> logger)
    {
        var appConfig = configuration.GetSection("AzureAd").Get<MicrosoftIdentityOptions>();
        _app = ConfidentialClientApplicationBuilder.Create(appConfig.ClientId)
            .WithTenantId(appConfig.TenantId)
            .WithClientSecret(appConfig.ClientSecret)
            .Build();

        _configuration = configuration;
        _logger = logger;
    }

    public async Task<string> CallGET(string accessToken, string url)
    {
        _logger.LogDebug("CallGET -> {url}", url);
        HttpClient httpClient = await GetNewHttpClient(accessToken);
        var response = await httpClient.GetAsync(
            $"{_configuration.GetValue<string>("DownstramApi:BaseUrl")}{url}");
        return await ConvertResponse(response);
    }

    public async Task<string> CallPATCH(string accessToken, string url, string data)
    {
        _logger.LogDebug("CallPATCH -> {url} with {data}", url, data);
        HttpClient httpClient = await GetNewHttpClient(accessToken);
        var content = new StringContent(data, Encoding.UTF8, "application/json");
        var response = await httpClient.PatchAsync(
            $"{_configuration.GetValue<string>("DownstramApi:BaseUrl")}{url}", content);
        return await ConvertResponse(response);
    }

    public async Task<string> GetNewOnBehalfOfToken(string accessToken)
    {
        string[] scopes = { _configuration.GetSection("DownstramApi").GetValue<string>("Scopes") };
        UserAssertion userAssertion = new UserAssertion(accessToken, "urn:ietf:params:oauth:grant-type:jwt-bearer");
        var result = await _app.AcquireTokenOnBehalfOf(scopes, userAssertion).ExecuteAsync();
        return result.AccessToken;
    }

    private async Task<string> ConvertResponse(HttpResponseMessage response)
    {
        var content = await response.Content.ReadAsStringAsync();
        var url = response.RequestMessage.RequestUri;
        if (response.StatusCode != HttpStatusCode.OK)
        {
            _logger.LogError(
                "ConvertResponse -> request '{url}' did not return HTTP 200 but HTTP {statusCode}. This will throw an error.",
                url, response.StatusCode);
            throw new HttpRequestException($"{response.StatusCode}:{content}");
        }

        _logger.LogDebug("ConvertResponse -> {url} responded {response}.", url, content);
        return content;
    }

    private async Task<HttpClient> GetNewHttpClient(string accessToken)
    {
        HttpClient httpClient = new();
        httpClient.DefaultRequestHeaders.Authorization =
            new AuthenticationHeaderValue("Bearer", await GetNewOnBehalfOfToken(accessToken));
        return httpClient;
    }
}

---

In combination the whole "story" looks like this (in parts pseudo code)

IMPORTANT: This also involves websocket & the "analysis" registry logic. The registry is used for requesting status & result of started analysis via REST calls. I didn't share the involved code for this here. Let me know if you are interested.

[Authorize]
[ApiController]
[Produces("application/json")]
[Route("api/[controller]")]
[RequiredScope("access-as-user")]
public class AnalysisController : ControllerBase
{
    private readonly IAnalysisRegister _analysisRegister;
    private readonly DownstreamApiCaller _downstreamApi;
    private readonly ILogger _logger;
    private readonly WebSocketHandler _webSocketHandler;

    public AnalysisController(
        IAnalysisRegister analysisRegister,
        DownstreamApiCaller downstreamApi,
        ILogger<ModelController> logger,
        DefaultWebSocketHandler webSocketHandler)
    {
        _analysisRegister = analysisRegister;
        _downstreamApi = downstreamApi;
        _logger = logger;
        _webSocketHandler = webSocketHandler;
    }

    private static async Task<JObject> BackgroundAnalysis(
        Guid analysisId,
        string accessToken,
        int id,
        IServiceScopeFactory serviceScopeFactory)
    {
        using var scope = serviceScopeFactory.CreateScope();

        JObject analysisIdAsObject = JObject.FromObject(new { analysis = analysisId });
        var provider = scope.ServiceProvider;
        var analysisRegister = provider.GetRequiredService<IAnalysisRegister>();
        var downstreamApi = provider.GetRequiredService<DownstreamApiCaller>();
        var loggerFactory = provider.GetRequiredService<ILoggerFactory>();
        var logger = Serilog.Log.ForContext<AnalysisController>();
        User user = await downstreamApi.User(accessToken);
        var webSocketHandler = provider.GetRequiredService<DefaultWebSocketHandler>();
        try
        {
            await webSocketHandler.SendMessageAsync(user.GroupId, "analysis-started", analysisIdAsObject);
            Analyser analyser = provider.GetRequiredService<Analyser>();
            Result result = await analyser.Analyse(accessToken, modelId);

            await downstreamApi.CallPATCH(accessToken, $"foo/{id}", JObject.FromObject(new
            {
                // we can remove `analysisId` now since the analysis is done
                analysisId = JValue.CreateNull(),
                someValueInResult = result.valueThatTookVeryLongToCompute
            }));

            // now we inform the client that we are done
            JObject updated = JObject.FromObject(new
            {
                analysisId = analysisId,
                someValueInResult = result.valueThatTookVeryLongToCompute
            });
            await webSocketHandler.SendMessageAsync(user.GroupId, "analysis-done", updatedModel);
            analysisRegister.Remove(analysisId);
            return updatedModel;
        }
        catch (Exception ex)
        {
            logger.Warning("StartAnalysis({modelId}) -> analysis '{analysisId}' FAILED", id, analysisId);
            logger.Error(ex, "Background task failed.");
            await webSocketHandler.SendMessageAsync(user.GroupId, "analysis-failed", analysisIdAsObject);
            return null;
        }
    }

    [HttpPut("start/{id:int:min(1)}")]
    [EnableCors]
    [ProducesResponseType(StatusCodes.Status200OK)]
    public async Task<IActionResult> StartAnalysis(
        int id,
        [FromServices] IServiceScopeFactory serviceScopeFactory)
    {
        _logger.LogTrace("StartAnalysis({id}) -> START", id);

        Guid analysisId = Guid.NewGuid();
        string accessToken = this.AccessToken();
        await _downstreamApi.CallPATCH(accessToken, $"foo/{id}", JObject.FromObject(new { analysisId = analysisId }));

        Task<JObject> analysis = Task.Run(async () =>
        {
            return await BackgroundAnalysis(
                accessToken: accessToken,
                analysisId: analysisId,
                id: id,
                serviceScopeFactory: serviceScopeFactory);
        });

        _analysisRegister.Register(analysisId, analysis);

        _logger.LogDebug("StartAnalysis({id}) -> analysis started: {analysisId}", id, analysisId);
        _logger.LogTrace("StartAnalysis({id}) -> END", id);
        return Ok(analysisId);
    }

}

Some links I used to put all of this together, I honestly don't remember all of them... it was a bunch:

• https://learn.microsoft.com/en-us/graph/auth-v2-user#token-request
• https://radu-matei.com/blog/aspnet-core-websockets-middleware/