Pass --cfg=bootstrap for proc macros built by stage0 #83433

jyn514 · 2021-03-24T01:54:34Z

Cargo has a bug where it ignores RUSTFLAGS when building proc macro
crates (rust-lang/cargo#4423).
However, sometimes rustc_macro needs to have conditional
compilation when there are breaking changes to the libproc_macro API
(see for example #83363). Previously, this wasn't possible, because the
crate couldn't tell the difference between stage 0 and stage 1.

Another alternative is to unconditionally build rustc_macros with the
master libstd instead of the beta one (i.e. use --sysroot stage0-sysroot), but that led to strange and maddening errors:

error[E0460]: found possibly newer version of crate `std` which `synstructure` depends on
 --> compiler/rustc_macros/src/lib.rs:5:5
  |
5 | use synstructure::decl_derive;
  |     ^^^^^^^^^^^^
  |
  = note: perhaps that crate needs to be recompiled?
  = note: the following crate versions were found:
          crate `std`: /home/joshua/rustc2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-b3602c301b71cc3d.rmeta
          crate `synstructure`: /home/joshua/rustc2/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/libsynstructure-74ee66863479e972.rmeta
error[E0460]: found possibly newer version of crate `std` which `proc_macro2` depends on
  --> /home/joshua/.local/lib/cargo/registry/src/github.heygears.com-1ecc6299db9ec823/tracing-attributes-0.1.13/src/lib.rs:90:5
   |
90 | use proc_macro2::TokenStream;
   |     ^^^^^^^^^^^
   |
   = note: perhaps that crate needs to be recompiled?
   = note: the following crate versions were found:
           crate `std`: /home/joshua/rustc2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-b3602c301b71cc3d.rmeta
           crate `proc_macro2`: /home/joshua/rustc2/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/libproc_macro2-a83c1f01610c129e.rlib

r? @Mark-Simulacrum cc @jhpratt

Mark-Simulacrum · 2021-03-24T02:06:36Z

src/bootstrap/bin/rustc.rs

@@ -126,6 +130,13 @@ fn main() {
        }
    }

+    if is_proc_macro && stage == "0" {
+        // Cargo has a bug where it doesn't pass RUSTFLAGS to proc_macros:


I.. wouldn't call this a bug :) More like 'unexpected behavior'. rust-lang/cargo#4423 seems like the more appropriate issue to cite, though.

Oops, yes, that's the one I meant to link to. The issue title isn't quite right though - we're neither cross-compiling nor building a build script.

jyn514 · 2021-03-24T15:00:46Z

Hmm, this doesn't actually work - when I pass --all-targets to x.py, cargo doesn't pass --crate-type proc-macro :/ I'm not sure how to fix this.

#83363 (comment)

Mark-Simulacrum · 2021-04-04T02:45:48Z

I unfortunately likely won't have too much time to help debug that problem, but I suspect it's either the same problem as noted in the comments (i.e., target-specific magic) or perhaps a separate Cargo bug that could be fixed. I imagine it should be reproducible outside rustbuild though, in which case you could file a new issue on rust-lang/cargo :)

jyn514 · 2021-04-07T02:04:27Z

I suspect it's either the same problem as noted in the comments (i.e., target-specific magic) or perhaps a separate Cargo bug that could be fixed.

I don't think this is actually a bug - passing --crate-type=proc-macro breaks unit tests:

$ RUSTFLAGS=--crate-type=proc-macro cargo test -q
error: test failed, to rerun pass '--lib'

Caused by:
  could not execute process `/home/joshua/.local/lib/cargo/target/debug/deps/libproc_macro_example-65ddfb6f3c001ad1.so --quiet` (never executed)

I guess I could special case --crate-name rustc_macros instead? But that seems awfully fragile :/

src/bootstrap/bin/rustc.rs

Cargo ignores RUSTFLAGS when building proc macro crates. However, sometimes rustc_macro needs to have conditional compilation when there are breaking changes to the `libproc_macro` API (see for example tell the difference between stage 0 and stage 1. Another alternative is to unconditionally build rustc_macros with the master libstd instead of the beta one (i.e. use `--sysroot stage0-sysroot`), but that led to strange and maddening errors: ``` error[E0460]: found possibly newer version of crate `std` which `proc_macro2` depends on --> /home/joshua/.local/lib/cargo/registry/src/github.heygears.com-1ecc6299db9ec823/tracing-attributes-0.1.13/src/lib.rs:90:5 | 90 | use proc_macro2::TokenStream; | ^^^^^^^^^^^ | = note: perhaps that crate needs to be recompiled? = note: the following crate versions were found: crate `std`: /home/joshua/rustc2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-b3602c301b71cc3d.rmeta crate `proc_macro2`: /home/joshua/rustc2/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/libproc_macro2-a83c1f01610c129e.rlib ```

jyn514 · 2021-06-05T02:02:18Z

This should be ready to go; I tested by rebasing #83363 on top of this and running x.py check --stage 1 and x.py check --all-targets.

Mark-Simulacrum · 2021-06-06T22:06:50Z

@bors r+ rollup

bors · 2021-06-06T22:06:52Z

📌 Commit dc30258 has been approved by Mark-Simulacrum

…laumeGomez Rollup of 8 pull requests Successful merges: - rust-lang#83433 (Pass --cfg=bootstrap for proc macros built by stage0) - rust-lang#84940 (Don't run sanity checks for `x.py setup`) - rust-lang#85912 (Use `Iterator::any` and `filter_map` instead of open-coding them) - rust-lang#85965 (Remove dead code from `LocalAnalyzer`) - rust-lang#86010 (Fix two ICEs in the parser) - rust-lang#86040 (Fix display for search results) - rust-lang#86058 (Remove `_` from E0121 diagnostic suggestions) - rust-lang#86077 (Fix corrected example in E0759.md) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

rust-highfive assigned Mark-Simulacrum Mar 24, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 24, 2021

jyn514 changed the title ~~Pass --cfg=bootstrap for proc_macros built by stage0~~ Pass --cfg=bootstrap for proc macros built by stage0 Mar 24, 2021

jyn514 mentioned this pull request Mar 24, 2021

Implement new proc macro diagnostics API #83363

Closed

Mark-Simulacrum reviewed Mar 24, 2021

View reviewed changes

This comment has been minimized.

# to view

jyn514 force-pushed the cfg-bootstrap-macro branch 2 times, most recently from 37104e1 to 2535828 Compare March 24, 2021 02:21

crlf0710 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 23, 2021

Mark-Simulacrum reviewed May 5, 2021

View reviewed changes

src/bootstrap/bin/rustc.rs Show resolved Hide resolved

Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 5, 2021

crlf0710 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 22, 2021

jyn514 force-pushed the cfg-bootstrap-macro branch from 2535828 to dc30258 Compare June 5, 2021 02:02

jyn514 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 5, 2021

bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 6, 2021

bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Jun 6, 2021

GuillaumeGomez mentioned this pull request Jun 6, 2021

Rollup of 8 pull requests #86080

Merged

bors merged commit 85fec06 into rust-lang:master Jun 7, 2021

rustbot added this to the 1.54.0 milestone Jun 7, 2021

jyn514 deleted the cfg-bootstrap-macro branch June 7, 2021 06:42

est31 mentioned this pull request Sep 16, 2022

Pass --cfg=bootstrap for rustdoc for proc_macro crates #101921

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Pass --cfg=bootstrap for proc macros built by stage0 #83433

Pass --cfg=bootstrap for proc macros built by stage0 #83433

Uh oh!

jyn514 commented Mar 24, 2021 •

edited

Loading

Uh oh!

Mark-Simulacrum Mar 24, 2021

Uh oh!

jyn514 Mar 24, 2021

Uh oh!

This comment has been minimized.

jyn514 commented Mar 24, 2021

Uh oh!

Mark-Simulacrum commented Apr 4, 2021

Uh oh!

jyn514 commented Apr 7, 2021

Uh oh!

Uh oh!

jyn514 commented Jun 5, 2021 •

edited

Loading

Uh oh!

Mark-Simulacrum commented Jun 6, 2021

Uh oh!

bors commented Jun 6, 2021

Uh oh!

Uh oh!

Pass --cfg=bootstrap for proc macros built by stage0 #83433

Pass --cfg=bootstrap for proc macros built by stage0 #83433

Uh oh!

Conversation

jyn514 commented Mar 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mark-Simulacrum Mar 24, 2021

Choose a reason for hiding this comment

Uh oh!

jyn514 Mar 24, 2021

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

jyn514 commented Mar 24, 2021

Uh oh!

Mark-Simulacrum commented Apr 4, 2021

Uh oh!

jyn514 commented Apr 7, 2021

Uh oh!

Uh oh!

jyn514 commented Jun 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mark-Simulacrum commented Jun 6, 2021

Uh oh!

bors commented Jun 6, 2021

Uh oh!

Uh oh!

jyn514 commented Mar 24, 2021 •

edited

Loading

jyn514 commented Jun 5, 2021 •

edited

Loading