Surefire rerun failing tests not working

Question

I want to rerun a test that I know will fail cause I am trying to test the Surefire parameter for re-running the failing tests. I tried running Maven with these two commands neither of them works as expected

-Dsurefire.rerunFailingTestsCount=2 -Dtest=TestThatFails test

and

-Dsurefire.rerunFailingTestsCount=2 -Dtest=TestThatFails surefire:test

Here is part of pom.xml

<dependency>
    <groupId>org.apache.maven.surefire</groupId>
    <artifactId>surefire-api</artifactId>
    <version>2.19.1</version>
</dependency>
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>2.53.1</version>
</dependency>

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>

I was expecting that Surefire would restart the test after failure but Maven just throws this error, which I know how to solve but I want the test to be rerun.

Results :

Tests in error: 
  testA(selenium.services.TestThatWillFail): Element is not currently visible and so may not be interacted with(..)

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 55.060 s
[INFO] Finished at: 2016-11-24T12:58:02+01:00
[INFO] Final Memory: 18M/173M

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project eskn_selenium: There are test failures.

Answer 1

Although that is missing from the documentation, the parameter rerunFailingTestsCount was introduced in version 2.18 of the Maven Surefire Plugin, as mentioned in SUREFIRE-1087. Since you're using the default version of 2.12.4 (that comes from the Super POM), that option is not available.

Therefore, the fix is simply to update the Surefire version to a version that is at least 2.18; for example, the latest, which is currently 2.19.1:

<pluginManagement>
  <plugins>
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.19.1</version>
    </plugin>
  </plugins>
</pluginManagement>

Note that this parameter only works with JUnit 4+ (which is your case, since you have JUnit 4.12).

Answer 2

Instead of using the command line property -Dsurefire.rerunFailingTestsCount=2, you can also define it in the pom in the properties section

 <properties>
    <surefire.rerunFailingTestsCount>2</surefire.rerunFailingTestsCount>
 </properties>

Answer 3

Just to add to Wim Rutgeerts's answer - rerunFailingTestsCount must be in the configuration section, not in properties, like this:

<configuration>
    <rerunFailingTestsCount>2</rerunFailingTestsCount>
</configuration>

In my case with maven-surefire-plugin 2.19.1 it worked this way. When it was in properties it did not work.

Answer 4

Update for JUnit 5: Maven Surefire version 3.0.0-M4 or later now lets you use rerunFailingTestsCount system property when executing JUnit 5 tests.

Make sure to pass the below property when running your mvn clean stage:

-Dsurefire.rerunFailingTestsCount=3