In azure Devops, Run on unit Test on Multi-Agent, how to collect the code coverage file and Publish into Artifact

Question

In azure Devops, Run Unit Test(VSTest) on Multi-Agent, how to collect the code coverage file and Publish into Artifact. In pipeline show only test result .trx File and unable to view .coverage File.

YAML Template File:

# Unit Test Execution      
- job: UnitTests_Execution
  displayName : 'Unit Tests execution'
  dependsOn: 'UnitTest_Build'
  condition: and(succeeded(), ${{ and(eq(parameters.RunCondition, 'True'), eq(parameters.RunUnitTest, 'True')) }})

  variables:
    BuildPlatform: 'any cpu'
    BuildConfiguration: 'release'
  
  pool:    
    name: 'Self-Hosted Agent'
  strategy:
    parallel: 8
      
  steps:
    - checkout: none
    
    - task: DownloadPipelineArtifact@2
      displayName: 'Download Build Artifacts'
      inputs:
        artifactName: unitTests
        targetPath: '$(System.ArtifactsDirectory)/unitTests'
    
    - task: VSTest@2
      displayName: 'Unit Tests'
      inputs:
        testSelector: 'testAssemblies'
        testAssemblyVer2: |
          **\*test*.dll
          !**\*TestAdapter.dll
          !**\obj\**
        searchFolder: '$(System.ArtifactsDirectory)/unitTests'
        resultsFolder: '$(System.ArtifactsDirectory)\TestResults\'
        runOnlyImpactedTests: false
        vsTestVersion: '${{ parameters.VsVersion }}'
        # runInParallel: true
        codeCoverageEnabled: true
        testRunTitle: 'Unit Testing service layer'
        platform: '$(BuildPlatform)'
        configuration: '$(BuildConfiguration)'
        rerunFailedTests: true

    - task: PublishPipelineArtifact@1 
      displayName: 'Publish Artifact: Code Coverage'
      inputs:
        targetPath: '$(build.ArtifactStagingDirectory)/TestResults'
        artifact: CodeCoverageResult

Using Multi-Agent, Am tried to collect the code covearge File and Merge into single File and publish into Sonar Cloud Dashboard

Excepted: In pipeline log only show the Test result .trx File and not show .coverage File. But in Single Agent it was working am excepting to work on multi-Agent concept.

Answer 1

I can reproduce the same issue when using the multiple-agents jobs.

It seems that the Code Coverage option doesn't support in vstest task when using multiple-agents jobs.

The Pipeline runs will point to the same Test Run link and not execute the real test method.

To solve this issue, you can use the parameters to loop the jobs to run the multiple jobs.

Here is an example:

parameters:
- name: runtimes
  type: object
  default: [1,2,3,4,5,6,7,8]


jobs:
- ${{ each runtime in parameters.runtimes }}:

  - job: UnitTests_Execution_${{ runtime }}
    displayName : 'Unit Tests execution ${{ runtime }}'
    dependsOn: 'UnitTest_Build'
    condition: and(succeeded(), ${{ and(eq(parameters.RunCondition, 'True'), eq(parameters.RunUnitTest, 'True')) }})

    variables:
      BuildPlatform: 'any cpu'
      BuildConfiguration: 'release'
    
    pool:    
      name: 'Self-Hosted Agent'

        
    steps:
      - checkout: none
      
      - task: DownloadPipelineArtifact@2
        displayName: 'Download Build Artifacts'
        inputs:
          artifactName: unitTests
          targetPath: '$(System.ArtifactsDirectory)/unitTests'
      
      - task: VSTest@2
        displayName: 'Unit Tests'
        inputs:
          testSelector: 'testAssemblies'
          testAssemblyVer2: |
            **\*test*.dll
            !**\*TestAdapter.dll
            !**\obj\**
          searchFolder: '$(System.ArtifactsDirectory)/unitTests'
          resultsFolder: '$(System.ArtifactsDirectory)\TestResults\'
          runOnlyImpactedTests: false
          vsTestVersion: '${{ parameters.VsVersion }}'
          # runInParallel: true
          codeCoverageEnabled: true
          testRunTitle: 'Unit Testing service layer'
          platform: '$(BuildPlatform)'
          configuration: '$(BuildConfiguration)'
          rerunFailedTests: true

      - task: PublishPipelineArtifact@1 
        displayName: 'Publish Artifact: Code Coverage'
        inputs:
          targetPath: '$(build.ArtifactStagingDirectory)/TestResults'
          artifact: CodeCoverageResult

In this case, it will run 8 parallel jobs too. And it can show the expected behavior.

Here is the result: .trx file and .coverage file