Support registration of global extensions via ServiceLoader mechanism

[reinhapa]

Overview

Since we have over 2000 tests that we would have to annotate with @ExtendWith, it would be nice to just add also extensions using the META-INF/service/org.junit.jupiter.api.extension.Extension mechanism (later under JDK 9 via service within module-info).

Is there a good reason to not have this possibility?

Pull Requests

• Support registration of global Extensions via java.util.ServiceLoader #451

[marcphilipp]

There is a good reason: Tests should be easy to understand.

However, I feel your pain. Maybe we could make this an opt-in feature that can be turned on via a configuration parameter / system property.

@junit-team/junit-lambda What do you think?

[reinhapa]

Hmm, I see you're point there. How would a opt-in feature look like?

[marcphilipp]

You could, e.g., enable global extensions via a system property. That would give you at least an indication...

[bechte]

I still feel that we should not go in this direction:

• JUnit5 supports JUnit4 tests and test need not to be changed to be executed on the new platform. Therefore, a migration is simple.
• If the tests are migrated to the JUnit5 Jupiter programming model, then at least the @Test annotation needs to be migrated to the Jupiter version. Using meta-annotation it would be easy to define a project specific version of @Test that adds the required @ExtendWith meta-annotations and, therefore, activates the default extension(s) on the fly.

While adding (very) more complexity to the extension registration, I cannot see the huge advantages that shall come along with this proposal. Even worse, it makes it more complex as we might also have to add a mechanism to "unregister" a globally registered extension for certain test cases. This is not worth the effort.

[schauder]

I very often wished there was a feature like this. Most often the scenario was like this:

• I create a test that works just fine on its own
• I run it with the complete test suite with a gazillion other tests and my new test fails.
• I realize one of the other tests is missing a teardown/after logik and leave some dirty state behind it.

What I ended up doing is: manually splitting all test in two groups, run them, check if my new test fails. If yes, split the tests in two again ... until finally I identify one bad test. Rinse and repeat until all bad apples are found.

With a global extention registry I could write a service that checks for left over state after each test and get a list of tests that need fixing.

Therefore I vote for such a feature.

[sbrannen]

However, I feel your pain. Maybe we could make this an opt-in feature that can be turned on via a configuration parameter / system property.

@junit-team/junit-lambda What do you think?

The Spring TestContext Framework supports this for the TestExecutionListener and ContextCustomizerFactory APIs, and this can be extremely useful.

For example, Spring Boot and Spring Security can auto-magically contribute custom testing features without requiring the user to do anything other than add the spring-boot-test or spring-security-test JAR to the classpath.

But that automatic registration itself is only part of the picture. The more important piece of the puzzle is that users can upgrade to a new version of Spring Security Test and benefit from its new testing features (provided via extensions/listeners) without having to configure every single test class in the entire test suite to register the new Spring Security Test feature alongside the existing configuration for the previous version of Spring Security Test.

Thus, I am in favor of supporting this feature in general and at the very least as an opt-in only feature.

[sbrannen]

@junit-team/junit-lambda, and objections on pulling this in before 5.0 GA?

[sbrannen]

Tentatively slated for M5.

[reinhapa]

@sbrannen where would be the correct place to add the according documentation to?

[sbrannen]

In the Registering Extensions section of the User Guide.

[reinhapa]

@sbrannen I've updated my pull request with some documentation

[sbrannen]

Now slated for M4.

[sbrannen]

Resolved in 473aa1a.