Skip to content

Accepted convention for incurring dependencies on external files in macro expansion hurts reproducibility #75887

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
pikajude opened this issue Aug 24, 2020 · 0 comments
Labels
A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) A-reproducibility Area: Reproducible / deterministic builds O-NixOS Operating system: NixOS, https://nixos.org/ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@pikajude
Copy link

pikajude commented Aug 24, 2020

According to this thread https://internals.rust-lang.org/t/pre-rfc-add-a-builtin-macro-to-indicate-build-dependency-to-file/9242/2 and a few other sources I found online, the common way to force a module with a proc-macro to be rebuilt based on external changes (for example, an HTML template) is to generate an expansion that uses include_bytes!() with the path to the file.

This function in librustc_span adds the string content of every macro input to the resulting rmeta file, including macros that are then re-expanded, like this one in the askama-derive crate. That macro expansion becomes include_bytes!("/absolute/path/to/source.html");, then the actual byte literal that contains the file contents.

I am using the Nix language/build system here and the cargo2nix tool, which causes every crate to be built in a temporary directory (to maximize isolation from the rest of the system). If the tempdir names differ between two builds of a crate that otherwise logically yields identical code, the metadata hashes will differ. (Though I haven't tested it myself, I can imagine this also being a problem with a tool like sccache if two devs with different filepaths to some git checkout try to use the same build artifacts).

@jonas-schievink jonas-schievink added O-NixOS Operating system: NixOS, https://nixos.org/ A-reproducibility Area: Reproducible / deterministic builds labels Aug 24, 2020
@jieyouxu jieyouxu added A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 15, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) A-reproducibility Area: Reproducible / deterministic builds O-NixOS Operating system: NixOS, https://nixos.org/ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

3 participants