Support non-host target for cargo install

[raphaelcohn]

On some systems cargo install needs to build and install binaries which are slightly different target to itself. For instance, on an Alpine Linux system, I might be using a rustup-installed x86_64-unknown-linux-gnu build of rust with LT_INTERP changed with patchelf, but I want to then build and install binaries as x86_64-unknown-linux-musl. It'd be nice to either have a default host, like rustup set default-host, or to take a switch --target like cargo build. cargo install seems to ignore environment variables like CARGO_BUILD_TARGET, too.

This need is also likely on some Windows systems (running i686 and x86_64), multilib Linux systems (eg imagine if a rust binary makes use of a proprietary i686 lib on a 64-bit system), and systems where the user space is usually 32-bit (Solaris IIRC, x64_32 linux).

This need would be mitigated substantially for me personally if rustup deployed a musl-statically-linked toolchain...

[alexcrichton]

Seems reasonable to add --target, I suspect it wouldn't be too hard to add!

[raphaelcohn]

Cool. Having thought about this a bit more, the underlying issue (compiling for a different native host) affects build.rs build scripts, in 2 ways:-

• A --target needs to be passed to rustc (as a workaround last night, I set RUSTC to a wrapper script and passed this argument to anything looking like a dev dependency or build script)
• Cargo would need to change the value of the HOST environment variable when calling a build script
  Thus the changes are probably a bit more invasive. Rather than being a --target, it's actually a --host, and would useful to pass to cargo install, cargo build and possibly cargo rustc.

In a manner of thinking, this change would make cargo effectively a canadian cross, without all the awful horribleness that involves in c-land...

[alexcrichton]

I think Cargo should already be all set up for cross compilation (it happens quite a bit normall), the install target is just missing the flag to plumb it to the backend.

[raphaelcohn]

Not quite. Cargo does not pass --target to rustc when compiling build scripts. It assumes that the rustc it runs has a native target that matches that of the system it's running on and that will work. This assumption fails on multilib systems. On systems with more than one LT_INTERP. On systems that compile in a sysroot that isn't the same as the system one, because the system one can not produce reproducible builds as it's constantly being changed from one build to a next by the package manager (apt or apk or what have you). That is only likely to be the case when that rustc is a system compiler. Likewise the value of HOST.

This is a flawed approach. Over many years, and having done many, many cross-compilations and developed approaches to bootstrap entire linux setups from nothing more than gcc, binutils and sh (no, not even make), I've come to the view that it is essential that all compilations are thought of as cross-compilations. Assume nothing.

[alexcrichton]

Er.. sorry I'm not quite sure I follow :(

[anatol]

+1 for adding transparent cross-compilation feature to cargo.

Do I understand correctly that it intends to cover functionality currently provided by xargo?

[aturon]

@alexcrichton I think what @raphaelcohn is saying is that:

• We currently assume that build scripts are never cross-compiled, which seems like a reasonable assumption since you're going to run them on the same machine that built them.
• However, in practice there are situations where that "same machine" actually has different execution contexts in play, such that you need to compile the build script with a distinct target, even though you'll run it on the same machine. You'll be running it within a different context.

They're further suggesting that one approach to solving this problem is to pass --target down for build scripts, at least some of the time (obvious not in the usual case).

Does that make sense to you?

[alexcrichton]

Ah ok, that does sort of make sense. I don't really understand the rationale though because if you're running a compiler with a particular target then you should be able to run a build script with that same target ...

[raphaelcohn]

@aturon I think --target should be passed down in all cases.

@alexcrichton Yes you're right if I understand what you're saying. However, Cargo is relying on rustc using a default target - which turns out to be what it is compiled with. And that toolchain may not work correctly on the host itself. A classic example is a glibc-linked rustc running on Alpine Linux. It can itself compile, but the toolchain that implies will produce incorrect binaries that won't run. Being explicit, rather than implicit, is a good practice in most situations, and this is no different. This problem is trivially rectified by having Cargo accept a --host, which it can then pass as HOST when running build scripts and as --target to rustc when compiling them. It'd be even better if such a value could go into Cargo's config.

@anatol This isn't to replace xargo - an excellent tool - but it would play very nicely with it. However, making this change would make it far easier to cross-compile new toolchains when the difference is one of libc settings, location of libc, static vs dynamic, etc.

[alexcrichton]

A classic example is a glibc-linked rustc running on Alpine Linux

To be clear though, this is a hack. This is not how the compiler nor Cargo is supposed to work, and naturally bugs are arising. Also to be clear the real solution to this issue is not this issue itself but rather supporting an x86_64-unknown-linux-musl compiler.

My preference would be to not contort Cargo to work around local hacks in the compiler.

[raphaelcohn]

Why is that a hack? It's quite a valid thing to do. There is nothing about Linux, or, indeed, any Unix derivative, that makes it a hack to use more than one runtime linker / libc. Indeed, such a technique has been in use for decades one a vendor supplied binary-only code in the days before open source was so popular. Skype comes to mind. No different to using LD_PRELOAD in that regard.

There aren't any bugs here, but a mistaken assumption. And that is that the toolchain that Cargo was compiled for is the toolchain of the host. Even in the land of madness that is C compilers, they don't force this. It is not a contortion for Cargo, as a compiler driver, to pass explicit arguments to its compiler. And to allow explicit arguments to the build scripts. These are not contortions. They're actually good programming practice, because one is making defaults and assumptions explicit and changeable.

[aturon]

@raphaelcohn Would you be willing to work on an RFC for this change to the Cargo compilation model? That would give you an opportunity to spell out both the motivation and proposed de# greater detail, and get feedback from the community.