Skip to content

benchmark: disable concurrent recompilation+sweeping? #11683

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
bnoordhuis opened this issue Mar 4, 2017 · 4 comments
Closed

benchmark: disable concurrent recompilation+sweeping? #11683

bnoordhuis opened this issue Mar 4, 2017 · 4 comments
Labels
benchmark Issues and PRs related to the benchmark subsystem. discuss Issues opened for discussions and feedbacks.

Comments

@bnoordhuis
Copy link
Member

I've noticed that many benchmarks have lots of variance unless you pass --noconcurrent_sweeping --noconcurrent_recompilation which forces V8 to run the mark-sweep GC phase and the optimizing compiler on the main thread.

Does it make sense to run with those flags by default? Considerations:

  1. It makes the benchmarks more reliable.
  2. It's not representative of how node is normally run.
  3. Micro-benchmarks aren't representative of normal use either.
  4. Is the variance indicative of problems with V8's thread pool?

(I'd say the answer to 4. is no, variable latency is practically a given with thread pools, but I haven't extensively tested what impact different values for --v8-pool-size=... have.)

cc @nodejs/benchmarking

@bnoordhuis bnoordhuis added benchmark Issues and PRs related to the benchmark subsystem. discuss Issues opened for discussions and feedbacks. labels Mar 4, 2017
@joyeecheung
Copy link
Member

I am a bit worried that this could make the microbenchmarks even more micro and results in optimizations landing when they can cause slowdowns in real-word usage, or prevent optimizations from landing even they benefit real-world usage..

@kunalspathak
Copy link
Member

Having the flag will also make it difficult for other VMs to get comparable results with node-v8 which was one of the thing that #9615 addressed.

@hashseed
Copy link
Member

V8 has a --predictable flag to reduce performance flakiness at the cost of running somewhat slower, by turning off a set of concurrency features.

@Trott
Copy link
Member

Trott commented Jul 30, 2017

This issue has been inactive for sufficiently long that it seems like perhaps it should be closed. Feel free to re-open (or leave a comment requesting that it be re-opened) if you disagree. I'm just tidying up and not acting on a super-strong opinion or anything like that.

@Trott Trott closed this as completed Jul 30, 2017
agostbiro added a commit to NomicFoundation/hardhat that referenced this issue Mar 18, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
benchmark Issues and PRs related to the benchmark subsystem. discuss Issues opened for discussions and feedbacks.
Projects
None yet
Development

No branches or pull requests

5 participants