Support alteration of discovered tests

[smoyer64]

I'm not sure that's a great description of what I want to ask ... in #153 the following code is discussed:

package org.example;

public class TopLevel {

    static class StaticNested {}

    @Nested
    class Inner {}

    @Test
    void method1() {}

    @Test
    void method2(TestInfo testInfo) {}

    @TestFactory
    Stream<DynamicTest> dynamicTests() {
        return Stream.of(new DynamicTest("dynamic test", () -> {}));
    }

}

Maybe I'm missing something but it looks like @testfactory can only add tests to an existing test plan. In order to support repeating parameterized test methods, there needs to be a mechanism to alter previously discovered tests (in essence, turning them into test containers).

While I love that a @testfactory can creates completely dynamic tests, there's a pattern that's common to JUnit4 test runners that I think is also important. I'll use the JUnitParams extension (one of my favorites) as an example - the generic code for a test would be something like this:

@RunWith(JUnitParamsRunner.class)
public void SomeTest {

  @Test
  @Parameter("1", "2", "3")
  public void testThatSomethingHappens(int someParameterThatMatter) {
    ...
  }

}

The above JUnitParams test class would result in three tests being executed. If I run that test today using JUnit4, it will use the test name as the container name and add the values of the parameters to the enclosed tests. I don't see how the equivalent pattern could be realized with JUnit5 (obviously, I'm not an expert in the code yet).

From the digging I did into the ALPHA release, I was picturing the addition of something like a test discovery callback that could be called after each @test was found (or for each DynamicTest returned by a @testfactory?). And during discovery, these test would have to be mutable so they could be expanded.

Am I missing something obvious? How would the @junit-team/junit5 developers suggest coding this pattern?

[smoyer64]

I see that as a result of #271, a CompositeTestSource was created. This looks like it provides the required hierarchy described in my problem statement above, so I guess the real question is how to "discover" a test like the JUnitParams example above.

[smoyer64]

Issue #354 provides a PoC that resolves this issue.

[marcphilipp]

This has been superseded by test templates.