How can I dynamically assign test groups to test methods and @BeforeXXX and @AfterXXX annotated methods during runtime?

Question

My Requirement:

I have around 100 tests in my test suite.

I don't want to hardcode the group name in the tests source code.

I want to choose the tests that I want to run each time but want to pass that list from an external source (csv/json/xml, etc) to the TestNG program via a maven command.

I can parameterize in the source code if needed, something like below:

@Test(groups = {"param1toBeReadFromExternalFile"})
@BeforeClass(groups = {"param1toBeReadFromExternalFile"})

Is this something that is achievable in TestNG?

Reason for this requirement:

The requirement is for external teams (that don't have access to source code) to be able to tag selective tests from the 100 tests with a specific group name and run tests on the fly without the need to check-out and check-in code.

Answer 1

Here's how you do it.

1. You first build an implementation of org.testng.IAnnotationTransformer and org.testng.IAlterSuiteListener
2. Within this implementation's constructor, you read/parse the JSON/CSV/XML/YAML file that contains the method to group mapping.
3. Within the transform() method, you filter out methods that match the incoming method and then add the groups as found in the data source file from (2).
4. Within the alter() you read a JVM argument that hints at which groups to be filtered and then apply that filter accordingly.

Here's a full fledged sample that uses the Google Gson library for json parsing.

Here's my test class

package com.rationaleemotions.runtime;

import java.util.Arrays;
import java.util.stream.Collectors;
import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.Test;

public class DemoClass1 {

  @Test
  public void testMethod1() {
    printer();
  }

  @Test
  public void testMethod2() {
    printer();
  }

  private static void printer() {
    ITestResult itr = Reporter.getCurrentTestResult();
    System.err.println(itr.getMethod().getQualifiedName() + " belongs to the groups " +
        Arrays.stream(itr.getMethod().getGroups()).collect(Collectors.toList()));
  }
}

Here's how the listener would look like

package com.rationaleemotions.runtime;

import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import java.util.List;
import java.util.stream.StreamSupport;
import org.testng.IAlterSuiteListener;
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;
import org.testng.xml.XmlGroups;
import org.testng.xml.XmlRun;
import org.testng.xml.XmlSuite;

public class GroupChanger implements IAnnotationTransformer, IAlterSuiteListener {

  private JsonArray json;

  public GroupChanger() throws FileNotFoundException {
    String mapping = System.getProperty("mapping.file", "src/test/resources/file.json");
    if (!mapping.trim().isEmpty()) {
      json = JsonParser.parseReader(new FileReader(mapping))
          .getAsJsonArray();
    }
  }

  @Override
  public void alter(List<XmlSuite> suites) {
    String groupsToRun = System.getProperty("groups", "g1");
    if (groupsToRun.equalsIgnoreCase("*")) {
      //Execute everything. So don't add groups filtering in the suite file
      return;
    }
    for (XmlSuite suite: suites) {
      XmlGroups xmlGroups = new XmlGroups();
      XmlRun xmlRun = new XmlRun();
      for (String group : groupsToRun.split(",")) {
        xmlRun.onInclude(group);
      }
      xmlGroups.setRun(xmlRun);
      suite.setGroups(xmlGroups);
    }
  }

  @Override
  public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor,
      Method testMethod) {
    if (testMethod == null) {
      return;
    }
    if (json == null) {
      return;
    }
    String fqmn = testMethod.getDeclaringClass().getCanonicalName() + "." + testMethod.getName();
    StreamSupport.stream(json.spliterator(), true)
        .filter(each -> methodName(each).equalsIgnoreCase(fqmn))
        .findFirst()
        .ifPresent(each -> {
          System.err.println("Found " + each);
          annotation.setGroups(groups(each));
        });
  }

  private static String methodName(JsonElement each) {
    return each.getAsJsonObject().get("method").getAsString();
  }

  private static String[] groups(JsonElement each) {
    return each.getAsJsonObject().get("groupName").getAsString().split(",");
  }
}

Here's how the suite file would look like

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="groups_suite" verbose="2">
  <listeners>
    <listener class-name="com.rationaleemotions.runtime.GroupChanger"/>
  </listeners>
  <test name="groups_test" verbose="2">
    <classes>
      <class name="com.rationaleemotions.runtime.DemoClass1"/>
    </classes>
  </test>
</suite>

Here's how the json mapping would look like

[
  {
    "method": "com.rationaleemotions.runtime.DemoClass1.testMethod1",
    "groupName": "g1"
  },
  {
    "method": "com.rationaleemotions.runtime.DemoClass1.testMethod2",
    "groupName": "g2"
  }
]