-
Notifications
You must be signed in to change notification settings - Fork 13.4k
More systematic compile/run markup for "pass" UI tests #61712
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Labels
A-testsuite
Area: The testsuite used to check the correctness of rustc
C-enhancement
Category: An issue proposing an enhancement or a PR with one.
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
I've added: /// Only run tests that match any of these modes.
pub filter_mode: Option<Vec<Mode>>, to The thinking is that you pass A |
After some thought, perhaps filtering is not even necessary with |
This was referenced Jun 10, 2019
Centril
added a commit
to Centril/rust
that referenced
this issue
Jun 23, 2019
compiletest: Introduce `// {check,build,run}-pass` pass modes Pass UI tests now have three modes ``` // check-pass // build-pass // run-pass ``` mirroring equivalent well-known `cargo` commands. `// check-pass` will compile the test skipping codegen (which is expensive and isn't supposed to fail in most cases). `// build-pass` will compile and link the test without running it. `// run-pass` will compile, link and run the test. Tests without a "pass" annotation are still considered "fail" tests. Most UI tests would probably want to switch to `check-pass`. Tests validating codegen would probably want to run the generated code as well and use `run-pass`. `build-pass` should probably be rare (linking tests?). rust-lang#61755 will provide a way to run the tests with any mode, e.g. bump `check-pass` tests to `run-pass` to satisfy especially suspicious people, and be able to make sure that codegen doesn't breaks in some entirely unexpected way. Tests marked with any mode are expected to pass with any other mode, if that's not the case for some legitimate reason, then the test should be made a "fail" test rather than a "pass" test. Perhaps some secondary CI can verify this invariant, but that's not super urgent. `// compile-pass` still works and is equivalent to `build-pass`. Why is `// compile-pass` bad - 1) it gives an impression that the test is only compiled, but not linked, 2) it doesn't mirror a cargo command. It can be removed some time in the future in a separate PR. cc rust-lang#61712
bors
added a commit
that referenced
this issue
Jun 23, 2019
compiletest: Introduce `// {check,build,run}-pass` pass modes Pass UI tests now have three modes ``` // check-pass // build-pass // run-pass ``` mirroring equivalent well-known `cargo` commands. `// check-pass` will compile the test skipping codegen (which is expensive and isn't supposed to fail in most cases). `// build-pass` will compile and link the test without running it. `// run-pass` will compile, link and run the test. Tests without a "pass" annotation are still considered "fail" tests. Most UI tests would probably want to switch to `check-pass`. Tests validating codegen would probably want to run the generated code as well and use `run-pass`. `build-pass` should probably be rare (linking tests?). #61755 will provide a way to run the tests with any mode, e.g. bump `check-pass` tests to `run-pass` to satisfy especially suspicious people, and be able to make sure that codegen doesn't breaks in some entirely unexpected way. Tests marked with any mode are expected to pass with any other mode, if that's not the case for some legitimate reason, then the test should be made a "fail" test rather than a "pass" test. Perhaps some secondary CI can verify this invariant, but that's not super urgent. `// compile-pass` still works and is equivalent to `build-pass`. Why is `// compile-pass` bad - 1) it gives an impression that the test is only compiled, but not linked, 2) it doesn't mirror a cargo command. It can be removed some time in the future in a separate PR. cc #61712
Centril
added a commit
to Centril/rust
that referenced
this issue
Jun 28, 2019
…petrochenkov Add `--pass $mode` to compiletest through `./x.py` Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`. When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file. The modes are explained in rust-lang#61778: - `check` has the same effect as `cargo check` - `build` or `compile` have the same effect as `cargo build` - `run` has the same effect as `cargo run` On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`. cc rust-lang#61712 r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Jun 28, 2019
…petrochenkov Add `--pass $mode` to compiletest through `./x.py` Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`. When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file. The modes are explained in rust-lang#61778: - `check` has the same effect as `cargo check` - `build` or `compile` have the same effect as `cargo build` - `run` has the same effect as `cargo run` On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`. cc rust-lang#61712 r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Jun 28, 2019
…petrochenkov Add `--pass $mode` to compiletest through `./x.py` Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`. When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file. The modes are explained in rust-lang#61778: - `check` has the same effect as `cargo check` - `build` or `compile` have the same effect as `cargo build` - `run` has the same effect as `cargo run` On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`. cc rust-lang#61712 r? @petrochenkov
Centril
added a commit
to Centril/rust
that referenced
this issue
Jun 29, 2019
…petrochenkov Add `--pass $mode` to compiletest through `./x.py` Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`. When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file. The modes are explained in rust-lang#61778: - `check` has the same effect as `cargo check` - `build` or `compile` have the same effect as `cargo build` - `run` has the same effect as `cargo run` On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`. cc rust-lang#61712 r? @petrochenkov
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Labels
A-testsuite
Area: The testsuite used to check the correctness of rustc
C-enhancement
Category: An issue proposing an enhancement or a PR with one.
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
TLDR:
TODO:
More detailed description, motivation, etc.
cc @Centril
The text was updated successfully, but these errors were encountered: