Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

🛠️ Repo: Improving test isolation in Workflow #78

Open
matiassimon opened this issue Jan 28, 2024 · 1 comment · May be fixed by #79
Open

🛠️ Repo: Improving test isolation in Workflow #78

matiassimon opened this issue Jan 28, 2024 · 1 comment · May be fixed by #79
Labels
area: repository tooling status: accepting prs Please, send a pull request to resolve this! 🙏 type: feature New feature or request

Comments

@matiassimon
Copy link

Summary:

The current workflow runs all tests together in a single job, causing two main issues:

  1. Tracking Difficulty:

    • It's challenging to distinguish which examples are functioning correctly and which aren't.
  2. Dependency Interference:

    • Shared system dependencies might lead to misleading test results; an example might pass due to a dependency installed by a different example.

Proposed Solutions:

  1. Matrix Strategy:

    • Consider replacing Lerna with a matrix strategy in the workflow, iterating over each package's folder independently.
    strategy:
      matrix:
        folder:
          - example1
          - example2
          - example3
        node-version:
          - 16
          - 18
  2. Separate Repositories:

    • Alternatively, use individual repositories for each example, each with its own workflows. Add them as submodules to a main repository.
@matiassimon matiassimon linked a pull request Jan 28, 2024 that will close this issue
@JoshuaKGoldberg JoshuaKGoldberg changed the title Improving test isolation in Workflow 🛠️ Repo: Improving test isolation in Workflow Jul 4, 2024
@JoshuaKGoldberg
Copy link
Member

Oh! Sorry for taking so long to get to this @matiassimon! We'd just started working on Mocha in January (mochajs/mocha#5027) and have mostly been focusing on the main repo. But this examples repo needs love too - and we should have responded sooner. Sorry!

+1 to this issue as described. It's annoying having them run in fail-fast series and interfering with each other.

@JoshuaKGoldberg JoshuaKGoldberg added type: feature New feature or request status: accepting prs Please, send a pull request to resolve this! 🙏 area: repository tooling labels Jul 4, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
area: repository tooling status: accepting prs Please, send a pull request to resolve this! 🙏 type: feature New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants