Class Level Test Annotation does not allow the setting of priorrities for the methods

[baubakg]

TestNG Version

7.5

Expected behavior

We should be able to set the priority of a test method even if the Test annotation is on the class.

Actual behavior

The piorities we set are ignored

Is the issue reproducible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

In my example I have a test case that ha different results based on the order. I will ry and change the order so that testBBB is executed first. I use the listener to make sure that the priority for the other method is higher.

Test Sample

(Edited from the original ass I had used a wrong example):

@Test
public class TestsPriority2 {
    
    public static int value = 100;
    
    
    public void testAAA() {
        value-=20;
    }
    
    
    public void testBBB() {
        value*=2;
    }
    

}

Test

@Test
    public void testNormalTestMethod_ChangedPriority_classLevelTest() {
        TestsPriority2.value=100;

       
        // Rampup
        TestNG myTestNG = createTestNG();
        TestListenerAdapter tla = fetchTestResultsHandler(myTestNG);

        // Define suites
        XmlSuite mySuite = addSuitToTestNGTest(myTestNG, "Automated Suite External priority Checks Testing");

        // Add listeners
        mySuite.addListener(PriorityThing.class.getTypeName());

        // Create an instance of XmlTest and assign a name for it.
        XmlTest myTest = attachTestToSuite(mySuite, "Test Simple External priority Checks Tests");
       

        //Define packages
        final Class<TestsPriority2> l_testClass = TestsPriority2.class;
        myTest.setXmlClasses(Collections.singletonList(new XmlClass(l_testClass)));

        myTestNG.run();

        assertThat("We should have no failed methods", tla.getFailedTests().size(), is(equalTo(0)));

        assertThat("We should have no failed methods", tla.getPassedTests().size(), is(equalTo(2)));
       assertThat("One of the results should be of prriority 2", tla.getPassedTests().stream().anyMatch(t -> t.getMethod().getPriority()==2));
        
        
        assertThat("We should not have the values in alfabetical order", TestsPriority2.value, equalTo(180));
     
    }

Listener

public class PriorityThing implements IAnnotationTransformer , ITestListener{
    protected static Logger log = LogManager.getLogger();
    
    @Override
    public void onTestStart(ITestResult result) {
        log.info("testStart: method : {} : priority is {}", result.getName(), result.getMethod().getPriority());
    }

    protected static Logger log = LogManager.getLogger();
    
    @Override
    public void onStart(ITestContext context) {
        for (ITestNGMethod lt_testNGMethod : context.getSuite().getAllMethods()) {

            log.info("onStart : Method {} priority is {}",lt_testNGMethod.getMethodName(),lt_testNGMethod.getPriority());
        }
        
    }

    @Override
    public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor,
            Method testMethod) {
        
        
        if (testMethod != null && testMethod.getName().equals("testAAA")) {
            annotation.setPriority(2);
            log.info("transform : Changing priority. priority for {} is {}",testMethod.getName(),annotation.getPriority());
            
        }
    }

}

[baubakg]

I would like to add that it seems that the order of execution is a bit odd, maybe that explains the priority thing:

INFO  | [main] priority.PriorityThing (PriorityThing.java:22) - in suite start
INFO  | [main] priority.PriorityThing (PriorityThing.java:36) - onStart : Method testAAA priority is 0
INFO  | [main] priority.PriorityThing (PriorityThing.java:36) - onStart : Method testBBB priority is 0
INFO  | [main] priority.PriorityThing (PriorityThing.java:27) - testStart: method : testAAA : priority is 0
INFO  | [main] priority.PriorityThing (PriorityThing.java:48) - transform : Changing priority. priority for testAAA is 2
INFO  | [main] priority.PriorityThing (PriorityThing.java:27) - testStart: method : testBBB : priority is 0

It seems that in this case the Test listener starts before the annotationtransfomer. Normally it is the other way around.

[krmahadevan]

@baubakg - Please recheck this. I can't reproduce this

import org.testng.annotations.Test;

public class TestsPriority2 {

  public static int value = 100;

  @Test
  public void testAAA() {
    value -= 20;
  }

  @Test
  public void testBBB() {
    value *= 2;
  }
}

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;

public class PriorityThing implements IAnnotationTransformer {

  @Override
  public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor,
      Method testMethod) {
    if (testMethod != null && testMethod.getName().equals("testAAA")) {
      annotation.setPriority(2);
      System.err.printf("transform : Changing priority. priority for %s is %s%n",
          testMethod.getName(), annotation.getPriority());
    }
  }
}

import org.assertj.core.api.Assertions;
import org.testng.TestNG;
import org.testng.annotations.Test;

public class IssueTest {

  @Test
  public void runTest() {
    TestNG testng = new TestNG();
    testng.addListener(new PriorityThing());
    testng.setTestClasses(new Class[]{TestsPriority2.class});
    testng.setVerbose(2);
    testng.run();
    Assertions.assertThat(TestsPriority2.value).isEqualTo(180);
  }
}

Output

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
transform : Changing priority. priority for testAAA is 2
PASSED: testBBB
PASSED: testAAA

===============================================
    Command line test
    Tests run: 2, Failures: 0, Skips: 0
===============================================


===============================================
Command line suite
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================


===============================================
Default Suite
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================


Process finished with exit code 0

[baubakg]

Sorry @krmahadevan I mixed my examples. Hre is the test sample which fails th piority stting (I have also updated the original description):

@Test
public class TestsPriority2 {
    
    public static int value = 100;
    
    
    public void testAAA() {
        value-=20;
    }
    
    
    public void testBBB() {
        value*=2;
    }
    

}

[juherr]

It is a design issue: when you set the annotation on the class, there is no annotation on methods.
And you can't change priority on an annotation that doesn't exist.

[baubakg]

Thanks for the feedback @juherr . Ok. Do you by any chance have a work around for setting the priority or changing the order of execution of the methods in this example?