Overhaul keyword handling #134253

nnethercote · 2024-12-13T11:18:46Z

The compiler's list of keywords has some problems.

It contains several items that aren't keywords.
The order isn't quite right in a couple of places.
Some of the names of predicates relating to keywords are confusing.
rustdoc and rustfmt have their own (incorrect) versions of the keyword list.
AllKeywords is unnecessarily complex.

r? @jieyouxu

rustbot · 2024-12-13T11:18:51Z

HIR ty lowering was modified

cc @fmease

Some changes occurred in src/tools/rustfmt

cc @rust-lang/rustfmt

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

nnethercote · 2024-12-16T09:13:30Z

Best reviewed one commit at a time.

jieyouxu · 2024-12-16T09:15:58Z

@nnethercote I haven't looked at this in detail just yet, just a quick question: does this PR adjust how contextual keywords like safe are handled?

nnethercote · 2024-12-16T10:28:51Z

@nnethercote I haven't looked at this in detail just yet, just a quick question: does this PR adjust how contextual keywords like safe are handled?

No. The only potential behavioural changes are that some of the (non)keywords (e.g. kw::DollarCrate, kw::StaticLifetime) that were accepted by the is_* predicates no longer are. But that had no effect on any tests. None of the contextual keywords were accepted by the predicates. More details in the individual commit messages.

compiler/rustc_span/src/symbol.rs

compiler/rustc_ast/src/token.rs

petrochenkov · 2024-12-16T18:33:17Z

Summary: there are many changes here that I wouldn't want to merge.

jieyouxu · 2024-12-16T19:42:39Z

(Unassigning myself as petrochenkov is already looking at it)

nnethercote · 2024-12-17T02:37:15Z

@petrochenkov: your definition of "keyword" doesn't align with that given by the language Reference, or the Ferrocene spec, or the terminology used by the compiler's error messages. But I have zero interest in litigating that so I have removed the things you objected to. I kept used_keywords as is because I think it's just as good there as somewhere else. I kept the name is_any_keyword but I added comments explaining that you shouldn't use it.

@rustbot ready

nnethercote · 2024-12-18T02:57:37Z

Rebased and comments addressed.

@bors r=petrochenkov

bors · 2024-12-18T02:57:40Z

📌 Commit 9753bf4 has been approved by petrochenkov

It is now in the queue for this repository.

bors · 2024-12-18T06:30:14Z

☔ The latest upstream changes (presumably #134243) made this pull request unmergeable. Please resolve the merge conflicts.

It's a verbose reinvention of a range type, and can be cut down a lot.

`rustc_symbol` is the source of truth for keywords. rustdoc has its own implicit definition of keywords, via the `is_doc_keyword`. It (presumably) intends to include all keywords, but it omits `yeet`. rustfmt has its own explicit list of Rust keywords. It also (presumably) intends to include all keywords, but it omits `await`, `builtin`, `gen`, `macro_rules`, `raw`, `reuse`, `safe`, and `yeet`. Also, it does linear searches through this list, which is inefficient. This commit fixes all of the above problems by introducing a new predicate `is_any_keyword` in rustc and using it in rustdoc and rustfmt. It documents that it's not the right predicate in most cases.

`gen` is an edition-specific keyword used in unstable Rust, and so belongs with `try` (as `is_unused_keyword_conditional` indicates). Also, the cases in `is_unused_keyword_conditional` should be in alphabetical order, to match the keyword list. These changes don't affect the behaviour of any of the `Symbol::is_*` functions.

In particular, clarify which predicates apply to which keywords.

nnethercote · 2024-12-18T09:32:28Z

I rebased.

@bors r=petrochenkov

bors · 2024-12-18T09:32:31Z

📌 Commit 6de550c has been approved by petrochenkov

It is now in the queue for this repository.

…etrochenkov Overhaul keyword handling The compiler's list of keywords has some problems. - It contains several items that aren't keywords. - The order isn't quite right in a couple of places. - Some of the names of predicates relating to keywords are confusing. - rustdoc and rustfmt have their own (incorrect) versions of the keyword list. - `AllKeywords` is unnecessarily complex. r? `@jieyouxu`

…etrochenkov Overhaul keyword handling The compiler's list of keywords has some problems. - It contains several items that aren't keywords. - The order isn't quite right in a couple of places. - Some of the names of predicates relating to keywords are confusing. - rustdoc and rustfmt have their own (incorrect) versions of the keyword list. - `AllKeywords` is unnecessarily complex. r? ``@jieyouxu``

Rollup of 8 pull requests Successful merges: - rust-lang#133702 (Variants::Single: do not use invalid VariantIdx for uninhabited enums) - rust-lang#133926 (Fix const conditions for RPITITs) - rust-lang#134161 (Overhaul token cursors) - rust-lang#134253 (Overhaul keyword handling) - rust-lang#134394 (Clarify the match ergonomics 2024 migration lint's output) - rust-lang#134420 (refactor: replace &PathBuf with &Path to enhance generality) - rust-lang#134444 (Fix `x build --stage 1 std` when using cg_cranelift as the default backend) - rust-lang#134452 (fix(LazyCell): documentation of get[_mut] was wrong) r? `@ghost` `@rustbot` modify labels: rollup

Rollup of 11 pull requests Successful merges: - rust-lang#130786 ( mir-opt: a sub-BB of a cleanup BB must also be a cleanup BB in `EarlyOtherwiseBranch`) - rust-lang#133926 (Fix const conditions for RPITITs) - rust-lang#134161 (Overhaul token cursors) - rust-lang#134253 (Overhaul keyword handling) - rust-lang#134394 (Clarify the match ergonomics 2024 migration lint's output) - rust-lang#134399 (Do not do if ! else, use unnegated cond and swap the branches instead) - rust-lang#134420 (refactor: replace &PathBuf with &Path to enhance generality) - rust-lang#134436 (tests/assembly/asm: Remove uses of rustc_attrs and lang_items features by using minicore) - rust-lang#134444 (Fix `x build --stage 1 std` when using cg_cranelift as the default backend) - rust-lang#134452 (fix(LazyCell): documentation of get[_mut] was wrong) - rust-lang#134460 (Merge some patterns together) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#134253 - nnethercote:overhaul-keywords, r=petrochenkov Overhaul keyword handling The compiler's list of keywords has some problems. - It contains several items that aren't keywords. - The order isn't quite right in a couple of places. - Some of the names of predicates relating to keywords are confusing. - rustdoc and rustfmt have their own (incorrect) versions of the keyword list. - `AllKeywords` is unnecessarily complex. r? ```@jieyouxu```

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Dec 13, 2024

nnethercote marked this pull request as draft December 13, 2024 11:18

petrochenkov self-assigned this Dec 13, 2024

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 13, 2024

This comment was marked as resolved.

# to view

nnethercote force-pushed the overhaul-keywords branch from b3dc14a to 0bbbe89 Compare December 16, 2024 05:59

nnethercote marked this pull request as ready for review December 16, 2024 05:59

nnethercote force-pushed the overhaul-keywords branch from 0bbbe89 to a6679b4 Compare December 16, 2024 09:08

rustbot assigned jieyouxu and unassigned petrochenkov Dec 16, 2024

jieyouxu added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 16, 2024

petrochenkov self-assigned this Dec 16, 2024

petrochenkov reviewed Dec 16, 2024

View reviewed changes

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 16, 2024

jieyouxu removed their assignment Dec 16, 2024

nnethercote force-pushed the overhaul-keywords branch from a6679b4 to 6a69d7b Compare December 17, 2024 02:32

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 17, 2024

rustbot added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Dec 17, 2024

nnethercote force-pushed the overhaul-keywords branch from 6a69d7b to 9753bf4 Compare December 18, 2024 02:56

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 18, 2024

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 18, 2024

nnethercote added 4 commits December 18, 2024 20:21

Simplify AllKeywords.

64abe8b

It's a verbose reinvention of a range type, and can be cut down a lot.

Improve comments on Keywords.

6de550c

In particular, clarify which predicates apply to which keywords.

nnethercote force-pushed the overhaul-keywords branch from 9753bf4 to 6de550c Compare December 18, 2024 09:32

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 18, 2024

jieyouxu mentioned this pull request Dec 18, 2024

Rollup of 8 pull requests #134456

Closed

jieyouxu mentioned this pull request Dec 18, 2024

Rollup of 8 pull requests #134458

Closed

jieyouxu mentioned this pull request Dec 18, 2024

Rollup of 11 pull requests #134470

Merged

bors merged commit 0a2d708 into rust-lang:master Dec 18, 2024
6 checks passed

rustbot added this to the 1.85.0 milestone Dec 18, 2024

nnethercote deleted the overhaul-keywords branch December 18, 2024 20:33

nnethercote mentioned this pull request May 22, 2025

Rename kw::Empty as sym::empty. #141376

Merged

Overhaul keyword handling #134253

Overhaul keyword handling #134253

Uh oh!

Conversation

nnethercote commented Dec 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Dec 13, 2024

Uh oh!

This comment was marked as resolved.

nnethercote commented Dec 16, 2024

Uh oh!

jieyouxu commented Dec 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nnethercote commented Dec 16, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

petrochenkov commented Dec 16, 2024

Uh oh!

jieyouxu commented Dec 16, 2024

Uh oh!

nnethercote commented Dec 17, 2024

Uh oh!

nnethercote commented Dec 18, 2024

Uh oh!

bors commented Dec 18, 2024

Uh oh!

bors commented Dec 18, 2024

Uh oh!

nnethercote commented Dec 18, 2024

Uh oh!

bors commented Dec 18, 2024

Uh oh!

Uh oh!

Uh oh!

nnethercote commented Dec 13, 2024 •

edited

Loading

jieyouxu commented Dec 16, 2024 •

edited

Loading