How to skip a Unit Test at runtime?

Question

Thanks in advance!

We have some Automation tests using the selenium web driver which are great and provide a really good regression pack.

The problem is now we have feature toggles in our code. So I need to say ignore these tests unless that feature toggle is turned On/ Off. I can't find anything really searching Google.

Ideally I don't want a 'if' statement at the top of the Feature tests but it looks like it's going to be the main way. My initial thoughts where to create a custom attribute

public class IsFeatureFlagTurnedOn : Attribute
{
   public IsFeatureFlagTurnedOn(string featureToggleName)
   {
      FeatureToggleName = featureToggleName;
   }
   public string FeatureToggleName {get;}
}

public class MyTests 
{
   [TestMethod]
   [IsFeatureFlagTurnedOn("MyFeature1")]
   public void ItShould()
   {
      // only run if MyFeature1 is turned on
   }
}

I some how need to hook into the MSTest pipeline and say if this attribute is present and the logic for MyFeature1 is turned off then don't run this test - Looked at dynamically adding the [Ignore] but with no luck.

This is running through VSTS and I could use [TestCategories] but I'd have to keep updating the pipeline to which feature is turned on/off which I don't want to do.

Any help or suggestions would be great!

Answer 1

MSTest v2 now has a lot of extensibility points, and you can achieve this by extending the TestMethodAttribute. First we add two attribute arguments, a string for a property name and a Type that has the property. Then we override the Execute method and invoke the property via reflection. If the result is true, we'll execute the test as normal, otherwise we return an 'inconclusive` test result.

public class TestMethodWithConditionAttribute : TestMethodAttribute
{
    public Type ConditionParentType { get; set; }
    public string ConditionPropertyName { get; set; }

    public TestMethodWithConditionAttribute(string conditionPropertyName, Type conditionParentType)
    {
        ConditionPropertyName = conditionPropertyName;
        ConditionParentType = conditionParentType;
    }

    public override TestResult[] Execute(ITestMethod testMethod)
    {
        if (ConditionParentType.GetProperty(ConditionPropertyName, BindingFlags.Static | BindingFlags.Public)?.GetValue(null) is bool condiiton && condiiton)
        {
            return base.Execute(testMethod);
        }
        else
        {
            return new TestResult[] { new TestResult {  Outcome = UnitTestOutcome.Inconclusive } };
        }
    }
}

Now we can use our new attribute like this:

[TestClass]
public class MyTests
{
    [TestMethodWithCondition(nameof(Configuration.IsMyFeature1Enabled), typeof(Configuration))]
    public void MyTest()
    {
        //...
    }
}

public static class Configuration
{
    public static bool IsMyFeature1Enabled => false;
}

---

The above is a very generic solution. You could also customize it a little more to your particular use case to perhaps avoid quite so much verbosity in the attribute declaration:

public class TestMethodForConfigAttribute : TestMethodAttribute
{
    public string Name { get; set; }

    public TestMethodForConfigAttribute(string name)
    {
        Name = name;
    }

    public override TestResult[] Execute(ITestMethod testMethod)
    {
        if (IsConfigEnabled(Name))
        {
            return base.Execute(testMethod);
        }
        else
        {
            return new TestResult[] { new TestResult {  Outcome = UnitTestOutcome.Inconclusive } };
        }
    }

    public static bool IsConfigEnabled(string name)
    {
        //...
        return false;
    }
}

And use it like:

[TestClass]
public class MyTests
{
    [TestMethodForConfig("MyFeature1")]
    public void MyTest()
    {
        //...
    }
}

Answer 2

Based on my reading of this, you may need to use Assert.Inconclusive