UserId not propagating from DirectLineJS to Azure Log Analytics

Question

Describe the problem

I have an angular application that uses DirectLineJS and connects to an Azure Bot (net 6 app) to get a Token and start conversation with a bot. I also configured the bot diagnostics to send QnA data to a log analytics workspace. However, userId is not being populated in the kusto reports.

Version

• For Frontend (Angular App): Using DirectLineJS version 0.15.1
• For Backend App using following packages:

        <PackageReference Include="Microsoft.Bot.Builder.AI.QnA" Version="4.19.3" />
        <PackageReference Include="Microsoft.Bot.Builder.Dialogs" Version="4.19.3" />
        <PackageReference Include="Microsoft.Bot.Builder.Integration.AspNet.Core" Version="4.19.3" />

This is how I'm getting the token from the Angular App:

getToken(iframeQueryParams: IFrameQueryParams): Observable<FCChatTokenDTO> {
    return this.http.get<FCChatTokenDTO>(
      `${environment.fcChatbotAPIUrl}/api/token?userId=${iframeQueryParams.userId}&divisionId=${iframeQueryParams.divisionId}`
    );
  }

And this is the method I use to send message to the bot:

  sendMessage(text: string) {
    const msg = {
      from: { id: this.userId, name: 'Test name' },
      type: 'message',
      text,
    } as FCChatMessage;
    return this.directline.postActivity(msg);
  }

this.userId variable is set to a user following direct line convention starting with dl_ prefix. Something like dl_123456

When I navigate to my Log Analytics Workspace and query for question & answer data like this:

// All QnA Traffic
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where OperationName=="CustomQuestionAnswering QueryKnowledgebases" // This OperationName is valid for custom question answering enabled resources
| sort by TimeGenerated desc

I get the following properties. Notice userId is empty:

{
"kbId": "FCKB",
"userId": "",
"question": "how do I create an account?",
"id": "2609",
"answer": "To create a deduction code for a health savings account (HSA), go to the SETUP tab > Payroll > Payroll Deductions. Click Add. Click the Add button for Health Savings Account (Single) or Health Savings Account (Family). Click OK in the confirmation window. Click the X in the top-right corner to close the Add Deduction panel. To make changes, click the Edit icon (pencil) next to the new code, and make your changes. \n\nThe Health Savings Account (Single) or Health Savings Account (Family) code is used for a pre-tax flat amount deduction of an employee's wages that they can opt-in to set aside to go into an HSA, which can pay for certain qualified medical expenses, such as copayments, deductibles, etc. For the Family option, the taxable wage limit is per family.",
"score": "68.37",
"source": "Q&As for earnings and deductions for FCKB.xlsx",
"isQnaOverDocumentRequest": "True",
"isAnswerFromDocumentText": "Microsoft.CognitiveServices.QnAMaker.Runtime.Models.QnASearchResult",
"azure-Resource-Id": "/subscriptions/XXXXXXXXXXX/resourcegroups/qna-bot-dev/providers/microsoft.cognitiveservices/accounts/fc-bot-language",
"apim-request-id": "XXXXXXXXX"
}

This is the method I use in my Azure Bot dotnet 6 app:

        public async Task<ActionResult<ChatConfig>> token([FromQuery] string? userId,[FromQuery] int? divisionId)
        {
            if (!string.IsNullOrWhiteSpace(_configuration["DirectLineSecret"]))
            {
                _DirectLineSecret = _configuration["DirectLineSecret"];
            }

            HttpClient client = new HttpClient();

            HttpRequestMessage request = new HttpRequestMessage(
                HttpMethod.Post,
                $"https://directline.botframework.com/v3/directline/tokens/generate");

            request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", _DirectLineSecret);

            if(string.IsNullOrEmpty(userId))
            {
                userId = Guid.NewGuid().ToString();
            }

            if (divisionId == null)
            {
                divisionId = 0;
            }

            //Important: User Id value needs to start with dl_ prefix, otherwise request will fail with 401
            var userParam = $"dl_u_{userId}_d_{divisionId}";

            request.Content = new StringContent(
                Newtonsoft.Json.JsonConvert.SerializeObject(
                    new { User = new { Id = userParam } }),
                    Encoding.UTF8,
                    "application/json");

            var response = await client.SendAsync(request);
            string token = String.Empty;
            string conversationId = String.Empty;

            if (response.IsSuccessStatusCode)
            {
                var body = await response.Content.ReadAsStringAsync();
                var directLineToken = JsonConvert.DeserializeObject<DirectLineToken>(body);
                token = directLineToken.token;
                conversationId = directLineToken.conversationId;
            }

            var config = new ChatConfig()
            {
                token = token,
                userId = userParam,
                conversationId = conversationId
            };

            return Ok(config);
        }

Expected behavior

I would expect userId to be populated in the result of my kusto query since I'm generating a token passing a correct userId and also passing the user id via the directLine.postActivity function. Why is userId not being populated?

I tried using the directLine.setUserId but I get the error message from below:

        if (userIdFromToken) {
            return console.warn('DirectLineJS: user id is already set in token, will ignore this user id.');