stdlib #[bench]s are (possibly) never run #15842
Labels
A-testsuite
Area: The testsuite used to check the correctness of rustc
Comments
huonw
changed the title
|
I believe they are all currently compiled, but never run unless you pass Running a benchmark for one iteration seems quite reasonable! |
|
Triage: don't think anything has changed here. |
huonw
added a commit
to huonw/rust
that referenced
this issue
May 2, 2015
E.g. if `foo.rs` looks like
#![feature(test)]
extern crate test;
#[bench]
fn bar(b: &mut test::Bencher) {
b.iter(|| {
1
})
}
#[test]
fn baz() {}
#[bench]
fn qux(b: &mut test::Bencher) {
b.iter(|| {
panic!()
})
}
Then
$ rustc --test foo.rs
$ ./foo
running 3 tests
test baz ... ok
test qux ... FAILED
test bar ... ok
failures:
---- qux stdout ----
thread 'qux' panicked at 'explicit panic', bench.rs:17
failures:
qux
test result: FAILED. 2 passed; 1 failed; 0 ignored; 0 measured
$ ./foo --bench ba
running 2 tests
test baz ... ignored
test bar ... bench: 97 ns/iter (+/- 74)
test result: ok. 0 passed; 0 failed; 1 ignored; 1 measured
In particular, the two benchmark are being run as tests in the default
mode.
This helps for the main distribution, since benchmarks are only run with
`PLEASE_BENCH=1`, which is rarely set (and never set on the test bots),
and helps for code-coverage tools: benchmarks are run and so don't count
as dead code.
Fixes rust-lang#15842.
bors
added a commit
that referenced
this issue
May 3, 2015
E.g. if `foo.rs` looks like
#![feature(test)]
extern crate test;
#[bench]
fn bar(b: &mut test::Bencher) {
b.iter(|| {
1
})
}
#[test]
fn baz() {}
#[bench]
fn qux(b: &mut test::Bencher) {
b.iter(|| {
panic!()
})
}
Then
$ rustc --test foo.rs
$ ./foo
running 3 tests
test baz ... ok
test qux ... FAILED
test bar ... ok
failures:
---- qux stdout ----
thread 'qux' panicked at 'explicit panic', bench.rs:17
failures:
qux
test result: FAILED. 2 passed; 1 failed; 0 ignored; 0 measured
$ ./foo --bench ba
running 2 tests
test baz ... ignored
test bar ... bench: 97 ns/iter (+/- 74)
test result: ok. 0 passed; 0 failed; 1 ignored; 1 measured
In particular, the two benchmark are being run as tests in the default
mode.
This helps for the main distribution, since benchmarks are only run with
`PLEASE_BENCH=1`, which is rarely set (and never set on the test bots),
and helps for code-coverage tools: benchmarks are run and so don't count
as dead code.
Fixes #15842.
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Jan 8, 2024
fix: try obligation of `IndexMut` when infer Closes rust-lang#15842. This issue arises because `K` is ambiguous if only inferred from `Index` trait, but is unique if inferred from `IndexMut`, but r-a doesn't use this info.
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
It seems to me that the only way for #15833 to be an issue is if we never run
#[bench]s anywhere during a PR merge (or on travis etc.).Maybe the test harness could run
#[bench]s as normal tests during a normal run (i.e. execute the code exactly once, not bothering to time it).The text was updated successfully, but these errors were encountered: