gcovr compare two reports

Question

I find it really difficult when refactoring huge codebase, changing tests etc. to find on my code coverage report what lines I'm no longer covering.

Is there any tool/way to get diff from two reports?

Answer 1

Gcovr currently has no support for showing coverage changes. This gives you roughly the following two options:

• You can use gcovr's JSON output and write your own script to compare the results from multiple different runs.
• You can use a different tool. For example, Henry Cox has created a fork of the lcov tool with interesting support for differential coverage. Many third party tools can show coverage changes, for example the Codecov.io service. If you're running some CI system, there might already be a feature of plugin to highlight coverage changes.

As an example of a script to process gcovr's JSON output, the following would alert you of lines that lost coverage:

#!/usr/bin/env python

import sys, json

def main(baseline_json: str, alternative_json: str):
  baseline = load_covered_lines(baseline_json)
  alternative = load_covered_lines(alternative_json)

  found_any_differences = False

  # compare covered lines for each source file
  for filename in sorted(baseline):
    difference = baseline[filename] - alternative[filename]

    if not difference:
      print(f"info: {filename}: ok")
      continue

    found_any_differences = True
    for lineno in sorted(difference):
      print(f"error: {filename}: {lineno}: not covered in {alternative_json}")

  if found_any_differences:
    sys.exit(1)


def load_covered_lines(gcovr_json_file: str) -> dict[str, set[int]]:
  # JSON format is documented at
  # <https://gcovr.com/en/stable/output/json.html#json-format-reference>

  with open(gcovr_json_file) as f:
    data = json.load(f)

  # The JSON format may change between versions
  assert data["gcovr/format_version"] == "0.3"

  covered_lines = dict()
  for filecov in data["files"]:
    covered_lines[filecov["file"]] = set(
      linecov["line_number"]
      for linecov in filecov["lines"]
      if linecov["count"] != 0 and not linecov["gcovr/noncode"]
    )

  return covered_lines


if __name__ == "__main__":
  main(*sys.argv[1:])

Example usage:

./old-configuration
gcovr --json baseline.json

find . -name '*.gcda' -exec rm {} +  # delete old coverage data

./new-configuration
gcovr --json alternative.json

python3 diffcov.py baseline.json alternative.json

Example output:

error: changed.c: 2: not covered in alternative.json
info: same.c: ok

You could create a similar script that also opens the original source files to create an annotated report with all coverage changes, similar to the textual reports generated by gcov.