Jacoco Not Showing Coverage for Robolectric Tests in Android Studio

Question

I'm facing an issue where the Jacoco code coverage results aren't showing up for unit tests that use Robolectric. The tests execute and pass successfully, but the Jacoco coverage seems to ignore files where Robolectric is utilized.

Environment:

• Android Studio: Giraffe | 2022.3.1 Patch 1
• Android Gradle Plugin (AGP): 7.2.0
• Gradle Version: 7.4.2
• Robolectric: 4.10.3

Project-Level build.gradle:

buildscript {
    val agp_version by extra("7.2.0")
}
plugins {
    id("com.android.application") version "7.2.0" apply false
    id("org.jetbrains.kotlin.android") version "1.9.0" apply false}

Module-Level build.gradle:

plugins {
    id("com.android.application")
    id("org.jetbrains.kotlin.android")
}

android {
    namespace = "com.example.jacocorobolelectric"
    compileSdk = 33

    defaultConfig {
        applicationId = "com.example.jacocorobolelectric"
        minSdk = 23
        targetSdk = 33
        versionCode = 1
        versionName = "1.0"
        testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            isMinifyEnabled = false
            proguardFiles(
                getDefaultProguardFile("proguard-android-optimize.txt"),
                "proguard-rules.pro"
            )
        }
    }
    compileOptions {
        sourceCompatibility = JavaVersion.VERSION_1_8
        targetCompatibility = JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = "1.8"
    }
}

dependencies {
    implementation("androidx.core:core-ktx:1.9.0")
    implementation("androidx.appcompat:appcompat:1.6.1")
    implementation("com.google.android.material:material:1.9.0")
    testImplementation("junit:junit:4.13.2")
    testImplementation ("org.robolectric:robolectric:4.10.3")
}

How can I address this issue?

Answer 1

The problem you're facing is a common one among Android developers who use Robolectric and Jacoco together. Jacoco is not always able to capture coverage for Robolectric tests out of the box due to the way Robolectric manipulates bytecode.

Here are steps to address this issue:

1. Apply the Jacoco Plugin: Ensure that you've applied the Jacoco plugin in your module-level build.gradle:

apply plugin: 'jacoco'

2. Configuration for Jacoco: Add a custom Jacoco configuration in your module-level build.gradle to specify the Jacoco version and other related settings:

jacoco {
    toolVersion = "0.8.7" // Update this to the latest available version
}

3. Test Task Configuration: Modify the testDebugUnitTest task to consider Robolectric classes for Jacoco code coverage. In the same module-level build.gradle:

tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
}

4. Robolectric Configuration: Add these JVM arguments to your Robolectric tests to ensure they're considered by Jacoco:

testOptions {
    unitTests {
        includeAndroidResources = true
        all {
            systemProperty 'jacoco-agent.destfile', 
                file("${project.buildDir}/jacoco/testDebugUnitTest.exec").absolutePath
        }
    }
}

5. Merge Execution Data: When generating coverage reports, ensure that you're merging all the generated .exec files. This will include data from both Robolectric tests and other unit tests.
6. Update Jacoco Version: Always use the latest version of Jacoco. They frequently release updates that address various issues. Check the official Jacoco release page for the latest version.

Once these configurations are set, re-run your tests, and then generate the Jacoco coverage report. Hopefully, you'll see coverage data for your Robolectric tests.

If you're still facing issues after making these changes, consider checking if there are any specific compatibility issues between the Jacoco version and Robolectric or Android Gradle Plugin versions you're using.

Answer 2

By default, Jacoco considers Robolectric files as external dependencies and excludes them from the code coverage. To include these files in the coverage, need to explicitly specify them in the Jacoco configuration.

Add the following to app-level build.gradle file:

jacocoTestReport {
    afterEvaluate {
        classDirectories = fileTree(
            dir: 'build/intermediates/classes/debug',
            excludes: [
                '**/R.class',
                '**/R$*.class',
                '**/BuildConfig.*',
                '**/Manifest*.*',
                '**/*Test*.*',
            ]
        )
    }
}

Answer 3

Here is what I'm doing to successfully capture coverage for both Robolectric tests and instrumented tests:

1. I'm using AGP 7.1.3, because I met this issue when using AGP 7.2.x
2. Add testCoverageEnabled = true to the module's debug build type:

buildTypes {
    debug {
        testCoverageEnabled = true
        ...
    }
    release {
        ...
    }
}

3. add config for Robolectric:

// For using Robolectric
tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
    jacoco.excludes = ['jdk.internal.*']
}

4. Set class directories, source directories, and execution data. Here is my entire jacoco.gradle: (apply it from module's gradle by apply from: 'jacoco.gradle' and then run gradle testDebugCoverage)

apply plugin: 'jacoco'
jacoco {
    toolVersion = "0.8.10"  // jacoco version
}

// For using Robolectric
tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
    jacoco.excludes = ['jdk.internal.*']
}
// Be invoked after the successful configuration of project
project.afterEvaluate {
    // Grab all build types and product flavors
    android.libraryVariants.all { variant ->
        def variantName = variant.name
        def unitTestTaskName = "test${variantName.capitalize()}UnitTest"
        def AndroidTestTaskName = "connected${variantName.capitalize()}AndroidTest"
        tasks.create(name: "test${variantName.capitalize()}Coverage", type: JacocoReport,
                dependsOn: ["$AndroidTestTaskName", "$unitTestTaskName"]) {
            group = "Reporting"
            description = "Generate Jacoco coverage reports for the ${variantName.capitalize()} build."
            // Set report format
            reports {
                html.enabled = true
                xml.enabled = true
                println(">>>>>>> outputLocation.path= $buildDir/reports/jacoco/test${variantName.capitalize()}Coverage/html")
            }

// Excludes the classes we don't need
def excludes = [
        '**/R.class',
        '**/R$*.class',
        '**/BuildConfig.*',
        '**/Manifest*.*',
        '**/*Test*.*',
        'android/**/*.*',
        'androidx/**/*.*'
]
// Java class
def javaClasses = fileTree(dir: variant.javaCompiler.destinationDir, excludes: excludes)
// Kotlin classes
def kotlinClasses = fileTree(dir: "${buildDir}/tmp/kotlin-classes/${variantName}", excludes: excludes)

// Class dirs
getClassDirectories().setFrom([javaClasses, kotlinClasses])

// Source dirs
getSourceDirectories().setFrom([
        "$project.projectDir/src/main/java",
        "$project.projectDir/src/${variantName}/java",
        "$project.projectDir/src/main/kotlin",
        "$project.projectDir/src/${variantName}/kotlin"
])
// Execution data
getExecutionData().setFrom(
        fileTree(dir: "$buildDir", include: ("outputs/code_coverage/debugAndroidTest/connected/**/*.ec")),
        fileTree(dir: "$buildDir", includes: ["outputs/unit_test_code_coverage/debugUnitTest/*.exec"]),
        fileTree(dir: "$buildDir", includes: ["jacoco/*.exec"])
)

My Environment:

• Android Studio Electric Eel | 2022.1.1 Patch 2
• Android Gradle Plugin(AGP): 7.1.3
• Gradle Version: 7.5
• Robolectric: 4.6.1
• Jacoco Version: 0.8.10

Hope it helps~