Serde with serde_derive fails to build in "alt" compiler on OS X #42967

SimonSapin · 2017-06-29T16:30:52Z

Steps to reproduce:

cargo new a
cd a
echo 'serde = {version = "1.0", features = ["serde_derive"]}' >> Cargo.toml
cargo build

Alternative steps to reproduce that will provides a suitable compiler:

# This branch is temporary,
# it will disappear once this issue is resolved and later be re-purposed
git clone https://github.com/servo/servo/ --branch rustup
cd servo
./mach cargo build -p serde

On OS X, with LLVM assertions disabled (like in rustc builds added in #39754, which are the default in Servo’s mach), rustc 1.20.0-nightly (69c65d2 2017-06-28) fails with:

error: dlopen(/private/tmp/a/target/debug/deps/libserde_derive-73114808ba8f6d5d.dylib, 1): Library not loaded: @rpath/libstd-82587804d4836e27.dylib
  Referenced from: /private/tmp/a/target/debug/deps/libserde_derive-73114808ba8f6d5d.dylib
  Reason: image not found
   --> /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs:261:1
    |
261 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile `serde`.

This problem does not occur with nightly-2017-06-29 from rustup, which has LLVM assertions enabled. (But perhaps that’s not the only difference between the two kinds of builds?)

The text was updated successfully, but these errors were encountered:

SimonSapin · 2017-06-29T16:34:29Z

perhaps that’s not the only difference between the two kinds of builds?

What’s RUST_CONFIGURE_ARGS="--enable-extended"?
https://github.com/rust-lang/rust/pull/39754/files#diff-354f30a63fb0907d4ad57269548329e3R75

SimonSapin · 2017-06-29T16:42:11Z

It’s also enabled in the "non-alt" build, so probably not relevant.

kennytm · 2017-06-29T16:49:32Z

Have you tried building the project in the same disk partition where you've installed rustc, i.e. inside /Users/...?

SimonSapin · 2017-06-29T16:51:40Z

Yes. With the second set of steps (and -v added), the error message looks like this:

   Compiling serde v1.0.8
     Running `rustc --crate-name serde /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs --crate-type lib --emit=dep-info,metadata -C codegen-units=4 -C debuginfo=2 --cfg 'feature="std"' --cfg 'feature="rc"' --cfg 'feature="derive"' --cfg 'feature="default"' --cfg 'feature="serde_derive"' -C metadata=b12802d89eb13761 -C extra-filename=-b12802d89eb13761 --out-dir /Users/simon/projects/servo/target/debug/deps -L dependency=/Users/simon/projects/servo/target/debug/deps --extern serde_derive=/Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib --cap-lints allow -W unused-extern-crates`
error: dlopen(/Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib, 1): Library not loaded: @rpath/libstd-82587804d4836e27.dylib
  Referenced from: /Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib
  Reason: image not found
   --> /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs:261:1
    |
261 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile `serde`.

kennytm · 2017-06-29T17:07:20Z

Does DYLD_PRINT_RPATHS=1 cargo build show anything special?

SimonSapin · 2017-06-29T17:10:04Z

@kennytm not as far I can tell.

kennytm · 2017-06-29T17:28:45Z

Okay... There are many other debugging options for dyld but I doubt if they are useful.

The problem is that @rpath/libstd-82587804d4836e27.dylib is not found, meaning @rpath/ is not expanded properly. I've checked libserde_derive-******.dylib does not have an LC_RPATH load command, meaning the @rpath is only supplied through the environment variable DYLD_LIBRARY_PATH set by Cargo.

I don't think the alt-build's and normal build's Cargo has any difference though.

Verify if rustc --print=sysroot is correct?

SimonSapin · 2017-06-29T17:38:11Z

./mach rustc --print=sysroot with and without llvm-assertions = true in .servobuild:

/Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc
/Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/rustc-nightly-x86_64-apple-darwin/rustc

Both contain:

bin		lib		manifest.in	share

kennytm · 2017-06-29T18:37:46Z

Both contains libstd-82587804d4836e27.dylib directly inside lib/? (not lib/rustlib/<target>/lib/)

SimonSapin · 2017-06-29T18:42:48Z

Almost, the -alt one has a different hash:

$ ls /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c*/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd*
/Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd-5e0ec722ea8df7e6.dylib
/Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd-82587804d4836e27.dylib

kennytm · 2017-06-30T04:39:03Z

Huh, that's even stranger. The @rpath should be resolved correctly. But the alt- rustc linked serde_derive using the libstd from the non-alt-build??

SimonSapin · 2017-06-30T06:06:00Z

Well it sounds unlikely to me that “from the other build” is really what’s going on, I’d bet on incorrect hashing. How is the hexadecimal part of libstd-82587804d4836e27.dylib determined? What base values are being hashed?

kennytm · 2017-06-30T06:17:43Z

IIRC rustc will just search for all libstd-*, it doesn't recompute the hash. Would it work if you move /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/ out of /Users/simon/tmp, and then do a cargo clean + build?

SimonSapin · 2017-06-30T08:51:31Z

I first hit the issue before the "non-alt" build was downloaded.

kennytm · 2017-06-30T09:31:18Z

I've tried to reproduce it myself and it seems $PATH plays a role in finding which libstd is linked. I can make it link to the correctly hashed libstd by:

"Install" it
Make sure $PATH does not contain ~/.cargo/bin, and includes «installed-path»/usr/local/bin
Build it. It should work.

Overriding $PATH without installing causes rustc not able to find libcore.

SimonSapin · 2017-06-30T11:38:58Z

What is "it" and what does "installing" mean?

kennytm · 2017-06-30T11:57:43Z

Run ./install.sh --destdir=«installed-path» after extracting the tarball (sorry was on mobile so did not check the exact instruction).

SimonSapin · 2017-06-30T12:02:30Z

What does this do besides copying the files?

Regardless, this is a regression, it only happens on OS X, and only with "alternate" builds…

Mark-Simulacrum · 2017-06-30T12:47:22Z

It looks like the problem is presumably that ./.servo/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd-5e0ec722ea8df7e6.dylib and ./.servo/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/lib/rustlib/x86_64-apple-darwin/lib/libstd-82587804d4836e27.dylib don't have the same hash.

One fairly clear problem is that Servo for some reason doesn't download rust-std from alt by-default AFAICT; having added some quick prints to the util script. That would lead to incompatibilities; this should be fixed.

looking for rustc at /Users/mark/Edit/servo/.servo/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/bin/rustc
Downloading {} https://s3.amazonaws.com/rust-lang-ci/rustc-builds-alt/69c65d29615c391c958ebf75dd65258ec23e175c/rustc-nightly-x86_64-apple-darwin.tar.gz
Downloading Rust compiler...
Downloading Rust compiler: 100.0%
Extracting Rust compiler...
Rust compiler ready.
Downloading {} https://s3.amazonaws.com/rust-lang-ci/rustc-builds/69c65d29615c391c958ebf75dd65258ec23e175c/rust-std-nightly-x86_64-apple-darwin.tar.gz
Downloading Host rust library for target x86_64-apple-darwin...
Downloading Host rust library for target x86_64-apple-darwin:  49.2%^Cmach interrupted by signal or user action. Stopping.

Mark-Simulacrum · 2017-06-30T12:58:22Z

I've filed servo/servo#17575 which should fix this.

SimonSapin · 2017-06-30T13:42:17Z

One fairly clear problem is that Servo for some reason doesn't download rust-std from alt by-default

Good catch! This fixes the problem, thank you.

This hopefully fixes rust-lang#42967 and rust-lang#43085.

…hton Enable profiler on "alternate" builds This hopefully fixes rust-lang#42967 and rust-lang#43085.

This reverts commit 6b52330. This is unnecessary now that rust-lang/rust#42967 is fixed by rust-lang/rust#43167. This migth be a fix for #17604

SimonSapin mentioned this issue Jun 29, 2017

Upgrade to rustc 1.20.0-nightly (69c65d296 2017-06-28) servo/servo#17561

Merged

Mark-Simulacrum mentioned this issue Jun 30, 2017

Download libstd from alt builders. servo/servo#17575

Closed

5 tasks

SimonSapin closed this as completed Jun 30, 2017

SimonSapin mentioned this issue Jul 6, 2017

Is (cross-)compiling with an "alt" compiler and "non-alt" standard library supported? #43085

Closed

SimonSapin added a commit to SimonSapin/rust that referenced this issue Jul 11, 2017

Enable profiler on "alternate" builds

a148f5b

This hopefully fixes rust-lang#42967 and rust-lang#43085.

SimonSapin mentioned this issue Jul 11, 2017

Enable profiler on "alternate" builds #43167

Merged

steveklabnik added a commit to steveklabnik/rust that referenced this issue Jul 13, 2017

Rollup merge of rust-lang#43167 - SimonSapin:alt-profiler, r=alexcric…

591d8a1

…hton Enable profiler on "alternate" builds This hopefully fixes rust-lang#42967 and rust-lang#43085.

SimonSapin mentioned this issue Aug 11, 2017

Metabug for Rust-related issues servo/servo#17012

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Serde with serde_derive fails to build in "alt" compiler on OS X #42967

Serde with serde_derive fails to build in "alt" compiler on OS X #42967

SimonSapin commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

kennytm commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

kennytm commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

kennytm commented Jun 29, 2017 •

edited

Loading

SimonSapin commented Jun 29, 2017

kennytm commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

kennytm commented Jun 30, 2017

SimonSapin commented Jun 30, 2017

kennytm commented Jun 30, 2017 •

edited

Loading

SimonSapin commented Jun 30, 2017

kennytm commented Jun 30, 2017

SimonSapin commented Jun 30, 2017

kennytm commented Jun 30, 2017

SimonSapin commented Jun 30, 2017

Mark-Simulacrum commented Jun 30, 2017

Mark-Simulacrum commented Jun 30, 2017

SimonSapin commented Jun 30, 2017

Serde with serde_derive fails to build in "alt" compiler on OS X #42967

Serde with serde_derive fails to build in "alt" compiler on OS X #42967

Comments

SimonSapin commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

kennytm commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

kennytm commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

kennytm commented Jun 29, 2017 • edited Loading

SimonSapin commented Jun 29, 2017

kennytm commented Jun 29, 2017

SimonSapin commented Jun 29, 2017

kennytm commented Jun 30, 2017

SimonSapin commented Jun 30, 2017

kennytm commented Jun 30, 2017 • edited Loading

SimonSapin commented Jun 30, 2017

kennytm commented Jun 30, 2017

SimonSapin commented Jun 30, 2017

kennytm commented Jun 30, 2017

SimonSapin commented Jun 30, 2017

Mark-Simulacrum commented Jun 30, 2017

Mark-Simulacrum commented Jun 30, 2017

SimonSapin commented Jun 30, 2017

kennytm commented Jun 29, 2017 •

edited

Loading

kennytm commented Jun 30, 2017 •

edited

Loading