Avoid unnecessary TokenTree to TokenStream conversions
#65455
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@bors try @rust-timer queue |
|
Awaiting bors try build completion |
bors
added a commit
that referenced
this pull request
Oct 15, 2019
…enStream-conversions, r=<try> Avoid unnecessary `TokenTree` to `TokenStream` conversions A `TokenStream` contains any number of `TokenTrees`. Therefore, a single `TokenTree` can be promoted to a `TokenStream`. But doing so costs two allocations: one for the single-element `Vec`, and one for the `Lrc`. (An `IsJoint` value also must be added; the default is `NonJoint`.) The current code converts `TokenTree`s to `TokenStream`s unnecessarily in a few places. This PR removes some of these unnecessary conversions, both simplifying the code and speeding it up. r? @petrochenkov
|
☀️ Try build successful - checks-azure |
|
Queued 881f05b with parent 237d54f, future comparison URL. |
petrochenkov
added
the
S-waiting-on-review
|
r=me after the perf is ready, with or without addressing #65455 (comment) |
petrochenkov
added
S-waiting-on-perf
|
Finished benchmarking try commit 881f05b, comparison URL. |
|
In case perf.rlo dies again the results are good. Screenshot
|
nnethercote
force-pushed
the
avoid-unnecessary-TokenTree-to-TokenStream-conversions
branch
from
08afe1f to
0fb82f3
Compare
|
I added @bors r=petrochenkov |
|
📌 Commit 0fb82f34d3ff58b28d5dcd21f89860b002313a3f has been approved by |
bors
added
the
S-waiting-on-bors
petrochenkov
removed
the
S-waiting-on-perf
|
☔ The latest upstream changes (presumably #65495) made this pull request unmergeable. Please resolve the merge conflicts. |
bors
added
S-waiting-on-author
The current code has this impl: ``` impl<T: Into<TokenStream>> iter::FromIterator<T> for TokenStream ``` If given an `IntoIterator<Item = TokenTree>`, it will convert each individual `TokenTree` to a `TokenStream` (at the cost of two allocations: a `Vec` and an `Lrc`). It will then merge those `TokenStream`s into a single `TokenStream`. This is inefficient. This commit changes the impl to this less general one: ``` impl iter::FromIterator<TokenTree> for TokenStream ``` It collects the `TokenTree`s into a single `Vec` first and then converts that to a `TokenStream` by wrapping it in a single `Lrc`. The previous generality was unnecessary; no other code needs changing. This change speeds up several benchmarks by up to 4%.
Because most of the call sites have an easier time working with a `TokenTree` instead of a `TokenStream`.
Likewise for `NestedMetaItem::tokens()`. Also, add `MetaItemKind::token_trees_and_joints()`, which `MetaItemKind::tokens()` now calls. This avoids some unnecessary `TokenTree` to `TokenStream` conversions, and removes the need for the clumsy `TokenStream::append_to_tree_and_joint_vec()`.
nnethercote
force-pushed
the
avoid-unnecessary-TokenTree-to-TokenStream-conversions
branch
from
0fb82f3 to
e4ec4a6
Compare
|
I rebased. @bors r=petrochenkov |
|
📌 Commit e4ec4a6 has been approved by |
bors
added
S-waiting-on-bors
tmandry
added a commit
to tmandry/rust
that referenced
this pull request
Oct 18, 2019
…Tree-to-TokenStream-conversions, r=petrochenkov Avoid unnecessary `TokenTree` to `TokenStream` conversions A `TokenStream` contains any number of `TokenTrees`. Therefore, a single `TokenTree` can be promoted to a `TokenStream`. But doing so costs two allocations: one for the single-element `Vec`, and one for the `Lrc`. (An `IsJoint` value also must be added; the default is `NonJoint`.) The current code converts `TokenTree`s to `TokenStream`s unnecessarily in a few places. This PR removes some of these unnecessary conversions, both simplifying the code and speeding it up. r? @petrochenkov
tmandry
added a commit
to tmandry/rust
that referenced
this pull request
Oct 18, 2019
…Tree-to-TokenStream-conversions, r=petrochenkov Avoid unnecessary `TokenTree` to `TokenStream` conversions A `TokenStream` contains any number of `TokenTrees`. Therefore, a single `TokenTree` can be promoted to a `TokenStream`. But doing so costs two allocations: one for the single-element `Vec`, and one for the `Lrc`. (An `IsJoint` value also must be added; the default is `NonJoint`.) The current code converts `TokenTree`s to `TokenStream`s unnecessarily in a few places. This PR removes some of these unnecessary conversions, both simplifying the code and speeding it up. r? @petrochenkov
tmandry
added a commit
to tmandry/rust
that referenced
this pull request
Oct 18, 2019
…Tree-to-TokenStream-conversions, r=petrochenkov Avoid unnecessary `TokenTree` to `TokenStream` conversions A `TokenStream` contains any number of `TokenTrees`. Therefore, a single `TokenTree` can be promoted to a `TokenStream`. But doing so costs two allocations: one for the single-element `Vec`, and one for the `Lrc`. (An `IsJoint` value also must be added; the default is `NonJoint`.) The current code converts `TokenTree`s to `TokenStream`s unnecessarily in a few places. This PR removes some of these unnecessary conversions, both simplifying the code and speeding it up. r? @petrochenkov
bors
added a commit
that referenced
this pull request
Oct 18, 2019
Rollup of 17 pull requests Successful merges: - #65016 (Always inline `mem::{size_of,align_of}` in debug builds) - #65197 (Prepare `MutVisitor`s to handle interned projections) - #65201 (Disable Go and OCaml bindings when building LLVM) - #65364 (Collect occurrences of empty blocks for mismatched braces diagnostic) - #65417 (Add more coherence tests) - #65434 (Add long error explanation for E0577) - #65455 (Avoid unnecessary `TokenTree` to `TokenStream` conversions) - #65472 (Use a sharded dep node to dep node index map) - #65480 (Speed up `LexicalResolve::expansion()`) - #65496 (properly document panics in div_euclid and rem_euclid) - #65508 (add option to ping llvm ice-breakers to triagebot) - #65511 (save-analysis: Nest tables when processing impl block definitions) - #65513 (reorder fmt docs for more clarity) - #65532 (doc: make BitSet intro more short) - #65540 (show up some extra info when t!() fails) - #65549 (Fix left/right shift typo in wrapping rotate docs) - #65552 (Clarify diagnostics when using `~` as a unary op) Failed merges: - #65471 (Add long error explanation for E0578) r? @ghost
tmandry
added a commit
to tmandry/rust
that referenced
this pull request
Oct 18, 2019
…Tree-to-TokenStream-conversions, r=petrochenkov Avoid unnecessary `TokenTree` to `TokenStream` conversions A `TokenStream` contains any number of `TokenTrees`. Therefore, a single `TokenTree` can be promoted to a `TokenStream`. But doing so costs two allocations: one for the single-element `Vec`, and one for the `Lrc`. (An `IsJoint` value also must be added; the default is `NonJoint`.) The current code converts `TokenTree`s to `TokenStream`s unnecessarily in a few places. This PR removes some of these unnecessary conversions, both simplifying the code and speeding it up. r? @petrochenkov
bors
added a commit
that referenced
this pull request
Oct 19, 2019
Rollup of 19 pull requests Successful merges: - #65016 (Always inline `mem::{size_of,align_of}` in debug builds) - #65197 (Prepare `MutVisitor`s to handle interned projections) - #65201 (Disable Go and OCaml bindings when building LLVM) - #65334 (Add long error explanation for E0575) - #65364 (Collect occurrences of empty blocks for mismatched braces diagnostic) - #65455 (Avoid unnecessary `TokenTree` to `TokenStream` conversions) - #65472 (Use a sharded dep node to dep node index map) - #65480 (Speed up `LexicalResolve::expansion()`) - #65493 (Add long error explanation for E0584) - #65496 (properly document panics in div_euclid and rem_euclid) - #65498 (Plugins deprecation: don’t suggest simply removing the attribute) - #65508 (add option to ping llvm ice-breakers to triagebot) - #65511 (save-analysis: Nest tables when processing impl block definitions) - #65513 (reorder fmt docs for more clarity) - #65532 (doc: make BitSet intro more short) - #65535 (rustc: arena-allocate the slice in `ty::GenericsPredicate`, not the whole struct.) - #65540 (show up some extra info when t!() fails) - #65549 (Fix left/right shift typo in wrapping rotate docs) - #65552 (Clarify diagnostics when using `~` as a unary op) Failed merges: - #65390 (Add long error explanation for E0576) - #65434 (Add long error explanation for E0577) - #65471 (Add long error explanation for E0578) r? @ghost
nnethercote
deleted the
avoid-unnecessary-TokenTree-to-TokenStream-conversions
branch
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A
TokenStreamcontains any number ofTokenTrees. Therefore, a singleTokenTreecan be promoted to aTokenStream. But doing so costs two allocations: one for the single-elementVec, and one for theLrc. (AnIsJointvalue also must be added; the default isNonJoint.)The current code converts
TokenTrees toTokenStreams unnecessarily in a few places. This PR removes some of these unnecessary conversions, both simplifying the code and speeding it up.r? @petrochenkov