Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

test_runner: emit test-only diagnostic warning #46540

Merged

Conversation

richiemccoll
Copy link
Contributor

@richiemccoll richiemccoll commented Feb 7, 2023

Fixes: #46448

This PR:

  • emits a test diagnostic when running tests with { only: true } or runOnlyTests(true) without the --test-only CLI flag.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added dont-land-on-v14.x needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Feb 7, 2023
@richiemccoll richiemccoll marked this pull request as ready for review February 7, 2023 10:02
@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch from 3ee8044 to e9bf7c1 Compare February 7, 2023 15:10
@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch 2 times, most recently from 8292f1b to 514fd0b Compare February 9, 2023 12:11
@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch from 514fd0b to 5669292 Compare February 9, 2023 12:28
Copy link
Member

@MoLow MoLow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, But I think we should emit a similar diagnostic when calling runOnly when no flag

@richiemccoll
Copy link
Contributor Author

richiemccoll commented Feb 10, 2023

I think we should emit a similar diagnostic when calling runOnly when no flag

@MoLow Are you referring to this scenario with runOnly below or something else? I've tested this locally and the diagnostic is there.

// running without the --test-only flag
const { test } = require('node:test');

test('top level test', async (t) => {
  t.runOnly(true);
  await t.test('this subtest is run');
});
TAP version 13
# Subtest: top level test
    # Subtest: this subtest is run
    ok 1 - this subtest is run
      ---
      duration_ms: 4.77543
      ...
    # 'only' and 'runOnly' require the --test-only command-line option.
    1..1
ok 1 - top level test
  ---
  duration_ms: 6.194772
  ...
1..1
# tests 1
# pass 1

@cjihrig
Copy link
Contributor

cjihrig commented Feb 11, 2023

I think we should emit a similar diagnostic when calling runOnly when no flag

I think @MoLow is saying to add the diagnostic when runOnly() is called, even if no tests are started. If this change is made, please ensure we don't end up in a situation where the diagnostic is included multiple times for the same test.

@richiemccoll
Copy link
Contributor Author

@cjihrig @MoLow Is that something you'd like to see as part of this PR or can that be tackled in a follow up commit?

@MoLow
Copy link
Member

MoLow commented Feb 13, 2023

Is that something you'd like to see as part of this PR or can that be tackled in a follow up commit?

A follow-up PR is ok

@MoLow
Copy link
Member

MoLow commented Feb 18, 2023

@richiemccoll this PR requires a rebase

@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch from 5669292 to 7696218 Compare February 20, 2023 09:29
@MoLow MoLow added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 20, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 20, 2023
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@MoLow MoLow added the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 21, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 21, 2023
@nodejs-github-bot nodejs-github-bot merged commit c90ea93 into nodejs:main Feb 21, 2023
@nodejs-github-bot
Copy link
Collaborator

Landed in c90ea93

MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@MoLow MoLow added the backport-open-v18.x Indicate that the PR has an open backport. label Feb 25, 2023
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
juanarbol pushed a commit that referenced this pull request Mar 3, 2023
PR-URL: #46540
Backport-PR-URL: #46839
Fixes: #46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@juanarbol juanarbol mentioned this pull request Mar 3, 2023
juanarbol pushed a commit that referenced this pull request Mar 5, 2023
PR-URL: #46540
Backport-PR-URL: #46839
Fixes: #46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this pull request Mar 13, 2023
PR-URL: #46540
Fixes: #46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this pull request Mar 14, 2023
PR-URL: #46540
Fixes: #46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
backport-open-v18.x Indicate that the PR has an open backport. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test_runner: warn if only is used without properly enabling only tests
6 participants