TestNG reports retried methods as separate test runs

Question

I started working with TESTNG for the first time few days back.

I implemented a retry analyzer using the IRetryAnalyzer interface which retries a failed test method for a maximum of 3 times.

I have a test method called retryInvoker() which fails twice and succeeds the third time.

The way TestNG reports this is 3 separate test runs, 2 of which failed and 1 succeeded, which is correct actually. Although I just wish to capture it as a single test run, which ultimately succeeded (if it did before the maximum allowed retries ended, which in this case were 3). Or even if it didn't succeed after 3 retries, I wish to report it as a single test run that failed instead of 4 separate test runs, all of which failed.

Any leads on this?

Answer 1

You could try the approach, suggested in this SO answer, removing the duplicates from the test context in case the same method is found either in failed or passed tests:

@Overrride
public void onFinish(ITestContext context) {
    Iterator<ITestResult> failedTestCases = context.getFailedTests().getAllResults().iterator();
    while (failedTestCases.hasNext()) {
        System.out.println("failedTestCases");
        ITestResult failedTestCase = failedTestCases.next();
        ITestNGMethod method = failedTestCase.getMethod();
        if (context.getFailedTests().getResults(method).size() > 1) {
            System.out.println("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
            failedTestCases.remove();
        } else {
            if (context.getPassedTests().getResults(method).size() > 0) {
                System.out.println(
                        "failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();
            }
        }
    }
}