User-experience outside of VS Code

[nickrobinson251]

The whole testitem framework is really cool, and we're considering adopting it for the tests of some or all of our packages (at RAI). I don't usually use VS Code, but i tried it out just for this feature, and i've got to say it made re-running test really nice!

If we're to be able to switch over to using TestItems, i think we'd want to contribute improvements to the user-experience when running outside of VS Code - e.g. when running tests from the Julia REPL in a terminal. Would PRs for such things be welcome?

For example, i'd love to figure out how a nice/easy way to select which testitems to re-run (rather than running a full ] test) -- perhaps adding support for regex-matching of testitem names (similar to what XUnit.jl supports)?

[davidanthoff]

Would PRs for such things be welcome?

Yes! The current non-VS Code experience really is the absolute bare minimum. I was primarily focused on the VS Code experience but felt I can't seriously release something that can't run in CI, so that was the goal for the first round. But there are soo many things one could improve about the non VS Code experience, and PRs for those would be fantastic. In an ideal world both the VS Code and the REPL/API experience would be on-par and super feature rich.

perhaps adding support for regex-matching of testitem names (similar to what XUnit.jl supports)

I think the filter option is probably good enough to somehow do that right now, but of course it is not super convenient, so adding more user-friendly APIs would be great.