-
Notifications
You must be signed in to change notification settings - Fork 13.2k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Pretty-printing adds additional parentheses, breaking the proc-macro reparse check #75734
Comments
Also causes dtolnay/async-trait#118. Rustc inserted unnecessary parentheses around the closure before calling async_trait, then lost all the location information on the source tokens and blamed the user for handwriting those unnecessary parentheses. |
It would break expressions with "virtual parentheses", like |
Cases like this could be caught when comparing the original and re-parsed token streams though, because the "virtual parentheses" ( So we can print without parentheses -> reparse -> compare, and then re-print with parentheses if the comparison fails. |
…etrochenkov Attach tokens to all AST types used in `Nonterminal` We perform token capturing when we have outer attributes (for nonterminals that support attributes - e.g. `Stmt`), or when we parse a `Nonterminal` for a `macro_rules!` argument. The full list of `Nonterminals` affected by this PR is: * `NtBlock` * `NtStmt` * `NtTy` * `NtMeta` * `NtPath` * `NtVis` * `NtLiteral` Of these nonterminals, only `NtStmt` and `NtLiteral` (which is actually just an `Expr`), support outer attributes - the rest only ever have token capturing perform when they match a `macro_rules!` argument. This makes progress towards solving rust-lang#43081 - we now collect tokens for everything that might need them. However, we still need to handle `#[cfg]`, inner attributes, and misc pretty-printing issues (e.g. rust-lang#75734) I've separated the changes into (mostly) independent commits, which could be split into individual PRs for each `Nonterminal` variant. The purpose of having them all in one PR is to do a single Crater run for all of them. Most of the changes in this PR are trivial (adding `tokens: None` everywhere we construct the various AST structs). The significant changes are: * `ast::Visibility` is changed from `type Visibility = Spanned<VisibilityKind>` to a `struct Visibility { kind, span, tokens }`. * `maybe_collect_tokens` is made generic, and used for both `ast::Expr` and `ast::Stmt`. * Some of the statement-parsing functions are refactored so that we can capture the trailing semicolon. * `Nonterminal` and `Expr` both grew by 8 bytes, as some of the structs which are stored inline (rather than behind a `P`) now have an `Option<TokenStream>` field. Hopefully the performance impact of doing this is negligible.
…etrochenkov Attach tokens to all AST types used in `Nonterminal` We perform token capturing when we have outer attributes (for nonterminals that support attributes - e.g. `Stmt`), or when we parse a `Nonterminal` for a `macro_rules!` argument. The full list of `Nonterminals` affected by this PR is: * `NtBlock` * `NtStmt` * `NtTy` * `NtMeta` * `NtPath` * `NtVis` * `NtLiteral` Of these nonterminals, only `NtStmt` and `NtLiteral` (which is actually just an `Expr`), support outer attributes - the rest only ever have token capturing perform when they match a `macro_rules!` argument. This makes progress towards solving rust-lang#43081 - we now collect tokens for everything that might need them. However, we still need to handle `#[cfg]`, inner attributes, and misc pretty-printing issues (e.g. rust-lang#75734) I've separated the changes into (mostly) independent commits, which could be split into individual PRs for each `Nonterminal` variant. The purpose of having them all in one PR is to do a single Crater run for all of them. Most of the changes in this PR are trivial (adding `tokens: None` everywhere we construct the various AST structs). The significant changes are: * `ast::Visibility` is changed from `type Visibility = Spanned<VisibilityKind>` to a `struct Visibility { kind, span, tokens }`. * `maybe_collect_tokens` is made generic, and used for both `ast::Expr` and `ast::Stmt`. * Some of the statement-parsing functions are refactored so that we can capture the trailing semicolon. * `Nonterminal` and `Expr` both grew by 8 bytes, as some of the structs which are stored inline (rather than behind a `P`) now have an `Option<TokenStream>` field. Hopefully the performance impact of doing this is negligible.
The following code:
is pretty-printed as:
The extra parenthesis completely change the structure of the re-parsed
TokenStream
(we now have tokens wrapped in aTokenTree::Delimited
), breaking the pretty-print and reparse check done during proc-macro expansion. This results in an instance of #43081.This behavior was introduced by #43742. The PR description states that is intended to "make pprust easier to use with programmatically constructed ASTs." - however, I can't find a concrete example of code that is affected by this.
I see a few possible ways of fixing this:
TokenStream
.ExprKind::Paren
when pretty-printing the AST. I'm not exactly certain what the consequences of doing this are. Do we ever end up pretty-printing code-generated AST nodes (e.g. from builtin macros), other than when explicitly requested via-Z unpretty=expanded
?If we could get it to work without breaking anything, solution 3 seems like the cleanest way of solving this.
cc @petrochenkov
The text was updated successfully, but these errors were encountered: