Can't get bearer token to work after authenticating with apple sign-in

Question

In my MAUI app, I use native Apple sign-in:

var webAuthenticatorResult = await AppleSignInAuthenticator.AuthenticateAsync(options);

From what I understand, Apple native doesn't have a ReturnUrl, like Google auth, so the AuthenticateAsync returns directly to the MAUI client. Here I unwrap the Claims from the id_token returned. Using the email and nameIdentifier, I create a JWT, like this:

private static string CreateToken(string email, string nameIdentifier)
{
    var issuer = "https://myappservice.azurewebsites.net/";
    var audience = "https://myappservice.azurewebsites.net/";
    var key = Encoding.ASCII.GetBytes("mySecretKey");

    var tokenDescriptor = new SecurityTokenDescriptor
    {
        Issuer = issuer,
        Audience = audience,
        Subject = new ClaimsIdentity(new[]
                            {
                                new Claim("Id", Guid.NewGuid().ToString()),
                                new Claim(JwtRegisteredClaimNames.Sub, nameIdentifier),
                                new Claim(JwtRegisteredClaimNames.Email, email),
                                new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString())
                            }),
        Expires = DateTime.UtcNow.AddMinutes(5),
        SigningCredentials = new SigningCredentials(new SymmetricSecurityKey(key), SecurityAlgorithms.HmacSha256Signature)
    };

    var tokenHandler = new JwtSecurityTokenHandler();
    var customAccessToken = tokenHandler.CreateToken(tokenDescriptor);
    return tokenHandler.WriteToken(customAccessToken);
}

My MAUI app contains a TokenHandler that makes sure to add this token to all outgoing requests, like this:

protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
    request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", TokenHolder.AccessToken);
    return await base.SendAsync(request, cancellationToken);
}

However, shortly after having created the JWT and when making a call to my API, it simply does not work. Commenting away 'RequireAuthorization' and it works just fine, but the Auth says no. This is my whole Auth setup in my appservice (also include the Google setup that works for the Google Webauth):

builder.Services.AddAuthentication(options =>
{
    options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;
    options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme;
    options.DefaultScheme = CookieAuthenticationDefaults.AuthenticationScheme;
}).AddJwtBearer(o =>
{
    o.TokenValidationParameters = new TokenValidationParameters
    {
        ValidIssuer = builder.Configuration["Jwt:Issuer"],
        ValidAudience = builder.Configuration["Jwt:Audience"],
        IssuerSigningKey = new SymmetricSecurityKey
        (Encoding.UTF8.GetBytes(builder.Configuration["Jwt:Key"])),
        ValidateIssuer = true,
        ValidateAudience = true,
        ValidateLifetime = false, //TODO: Denna bör vara true va?
        ValidateIssuerSigningKey = true
    };
}).AddCookie().AddGoogle(g =>
{
    g.ClientId = builder.Configuration["GoogleClientId"];
    g.ClientSecret = builder.Configuration["GoogleClientSecret"];
    g.SaveTokens = true;
});

Can you spot anything that I need to fix, or any pointers on how to attack the problem? I am developing in Visual Studio on Windows. I have a Mac Mini running Sonoma to which my iPhone is plugged in - hence I "cannot" (unless you have a good idea how) debug my backend (since it's not on any 'localhost') - I run my app on the iPhone towards the Azure appservice api.