Speed up bootstrap a little. #73352
Merged
Speed up bootstrap a little. #73352
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
(rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
|
I don't know that this performance matters that much - in practice, compiling is going to take a minute or two I think, at least. But I think it's a good cleanup regardless. I think it's fine to hard code profiler builtins and LLVM in bootstrap for the extra perf win as well. |
|
I does matter - running |
|
Yes, though I think those no-op builds will end up doing something similar to cargo metadata? So it's not as significant a win. Still, seconds shaved off of every build is good. |
This should run much faster. There are also some drive-by cleanups here to try to simplify things. Also, the paths for in-tree crates are now displayed as relative in `x.py test -h -v`.
|
OK, I have pushed an update that uses
|
|
r=me with the description updated to include the latest changes (profiler builtins and llvm in particular). |
|
@bors r=Mark-Simulacrum |
|
📌 Commit f17fd7b has been approved by |
bors
added
S-waiting-on-bors
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Jun 16, 2020
…mulacrum Speed up bootstrap a little. The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4. An overview of the changes: - Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config). - Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior. - Remove check for `_shim`, I believe all the `_shim` crates are now gone. - Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree. - Use relative paths in `./x.py test -h -v` output. - Some code cleanup (remove unnecessary `find_compiler_crates`, etc.). - Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output. - Some doc comments.
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Jun 16, 2020
…mulacrum Speed up bootstrap a little. The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4. An overview of the changes: - Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config). - Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior. - Remove check for `_shim`, I believe all the `_shim` crates are now gone. - Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree. - Use relative paths in `./x.py test -h -v` output. - Some code cleanup (remove unnecessary `find_compiler_crates`, etc.). - Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output. - Some doc comments.
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Jun 16, 2020
…mulacrum Speed up bootstrap a little. The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4. An overview of the changes: - Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config). - Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior. - Remove check for `_shim`, I believe all the `_shim` crates are now gone. - Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree. - Use relative paths in `./x.py test -h -v` output. - Some code cleanup (remove unnecessary `find_compiler_crates`, etc.). - Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output. - Some doc comments.
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Jun 16, 2020
…mulacrum Speed up bootstrap a little. The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4. An overview of the changes: - Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config). - Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior. - Remove check for `_shim`, I believe all the `_shim` crates are now gone. - Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree. - Use relative paths in `./x.py test -h -v` output. - Some code cleanup (remove unnecessary `find_compiler_crates`, etc.). - Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output. - Some doc comments.
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Jun 16, 2020
…mulacrum Speed up bootstrap a little. The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4. An overview of the changes: - Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config). - Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior. - Remove check for `_shim`, I believe all the `_shim` crates are now gone. - Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree. - Use relative paths in `./x.py test -h -v` output. - Some code cleanup (remove unnecessary `find_compiler_crates`, etc.). - Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output. - Some doc comments.
|
@bors rollup |
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Jun 19, 2020
…mulacrum Speed up bootstrap a little. The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4. An overview of the changes: - Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config). - Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior. - Remove check for `_shim`, I believe all the `_shim` crates are now gone. - Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree. - Use relative paths in `./x.py test -h -v` output. - Some code cleanup (remove unnecessary `find_compiler_crates`, etc.). - Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output. - Some doc comments.
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Jun 19, 2020
Rollup of 10 pull requests Successful merges: - rust-lang#72280 (Fix up autoderef when reborrowing) - rust-lang#72785 (linker: MSVC supports linking static libraries as a whole archive) - rust-lang#73011 (first stage of implementing LLVM code coverage) - rust-lang#73044 (compiletest: Add directives to detect sanitizer support) - rust-lang#73054 (memory access sanity checks: abort instead of panic) - rust-lang#73136 (Change how compiler-builtins gets many CGUs) - rust-lang#73280 (Add E0763) - rust-lang#73317 (bootstrap: read config from $RUST_BOOTSTRAP_CONFIG) - rust-lang#73350 (bootstrap/install.rs: support a nonexistent `prefix` in `x.py install`) - rust-lang#73352 (Speed up bootstrap a little.) Failed merges: r? @ghost
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Jul 8, 2020
…r=Mark-Simulacrum Fix x.py test for librustc crates. rust-lang#73352 introduced a bug where `x.py test src/librustc_ast` would fail to actually run the tests. The issue is that `krate` and `all_krates` were changed to return relative paths. This caused the code to do a test of "relative_path ends with absolute path" which is always false. The solution is to swap that around. The change to `Crate` isn't necessary, it just simplifies the code and makes it uniform with `CrateLibrustc`.
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Jul 8, 2020
…Mark-Simulacrum Fix x.py test for librustc crates. rust-lang#73352 introduced a bug where `x.py test src/librustc_ast` would fail to actually run the tests. The issue is that `krate` and `all_krates` were changed to return relative paths. This caused the code to do a test of "relative_path ends with absolute path" which is always false. The solution is to swap that around. The change to `Crate` isn't necessary, it just simplifies the code and makes it uniform with `CrateLibrustc`.
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The bootstrap script was calling
cargo metadata3 times (or 6 with-v). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like./x.py test -h -vgoes from about 3 seconds to 0.4.An overview of the changes:
cargo metadataonly once with--no-deps. Optional dependencies are filtered inin_tree_crates(handlingprofiler_builtinsandrustc_codegen_llvmwhich are driven by the config).metadata::buildwhen using-v. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior._shim, I believe all the_shimcrates are now gone.rustc_and*sanfortest::Crate::should_run, these are no longer dependencies in thetesttree../x.py test -h -voutput.find_compiler_crates, etc.).src/test/ui/...) in./x.py test -h -voutput.