Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Describe somewhere what the benchmarks measure, and how to measure yourself #275

Closed
RalfJung opened this issue Aug 21, 2018 · 1 comment
Closed

Comments

@RalfJung
Copy link
Member

I just realized that "baseline" and "clean" are the exact opposite of what I thought they were. "clean" sounds like it starts with a clean (empty) cache, and "baseline" like it would be the baseline, minimal cost of an incremental run with a hot cache. But actually it's the other way around.

Could the timing pages (like this one) explain this somewhere? Like a note at the bottom, or a tooltip in the actual table, or so. Having space for such an explanation would anyway be useful, as that would also be a good space to describe how one can run that same benchmark against a local rustc -- which usually is the first thing I need to do when I see a regression.

(I am still working on figuring out how to reproduce this "incremental clean" regression. Took me at least an hour, at this point, due to running into issues like rust-lang/cargo#5918.)

@Mark-Simulacrum
Copy link
Member

Fixed in b52f402.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants