Nightly installation fails for non-root user

[mattgodbolt]

The nightly builds are currrently failing as the cargo install as part of the nightly want to unconditionally place stuff in /etc:

install: copying file /opt/compiler-explorer/rust-nightly/share/doc/cargo/LICENSE-APACHE
install: copying file /opt/compiler-explorer/rust-nightly/share/doc/cargo/README.md
install: copying file /opt/compiler-explorer/rust-nightly/share/doc/cargo/LICENSE-THIRD-PARTY
install: copying file /etc/bash_completion.d/cargo
install: cannot create regular file '/etc/bash_completion.d/cargo': Permission denied
install: error: file creation failed. see logs at '/opt/compiler-explorer/rust-nightly/lib/rustlib/install.log'

the log file above says:

$ umask 022 && mkdir -p "/opt/compiler-explorer/rust-nightly/share/doc/cargo"
install: copying file /opt/compiler-explorer/rust-nightly/share/doc/cargo/LICENSE-THIRD-PARTY
$ install -m644 /tmp/rust-nightly-x86_64-unknown-linux-gnu/cargo/share/doc/cargo/LICENSE-THIRD-PARTY /opt/compiler-explorer/rust-nightly/share/doc/cargo/LICENSE-THIRD-PARTY
$ echo "file:/opt/compiler-explorer/rust-nightly/share/doc/cargo/LICENSE-THIRD-PARTY" >> "/opt/compiler-explorer/rust-nightly/lib/rustlib/manifest-cargo"
$ umask 022 && mkdir -p "/etc/bash_completion.d"
install: copying file /etc/bash_completion.d/cargo
$ install -m644 /tmp/rust-nightly-x86_64-unknown-linux-gnu/cargo/etc/bash_completion.d/cargo /etc/bash_completion.d/cargo
install: error: file creation failed. see logs at '/opt/compiler-explorer/rust-nightly/lib/rustlib/install.log'

[crumblingstatue]

@mattgodbolt
As a side-question: Is it not feasible to use a pre-built Rust (e.g. installed through rustup) for Compiler Explorer, instead of installing it from source?

[mattgodbolt]

@crumblingstatue - I'm not installing via source here (I don't think?). I'm running ./install.sh --without=rust-docs and then manaully patching the rpath to let me run the binaries directly by path.

CE installs to a shared NFS drive that all the various instances all mount. If rustup can be made to install somewhere "shared" like this, I can move to it. Importantly I need to be able to execute the full path to the rust compiler (/opt/compiler-explorer/rust/version/bin/rustc or similar) without any per-compiler environment variables being required.

[crumblingstatue]

I'm not installing via source here (I don't think?).

Sorry, I misunderstood.

CE installs to a shared NFS drive that all the various instances all mount. If rustup can be made to install somewhere "shared" like this, I can move to it. Importantly I need to be able to execute the full path to the rust compiler (/opt/compiler-explorer/rust/version/bin/rustc or similar) without any per-compiler environment variables being required.

Interesting. I'm not really knowledgeable on rustup and environments like this.
Perhaps @brson or @Diggsey could answer this.

Although... Now that I think about it, I think rustup just invokes ./install.sh when installing a toolchain, so this would be broken with rustup too.

[cuviper]

Probably related to --sysconfdir in #41853.

[Diggsey]

@crumblingstatue rustup doesn't use install.sh (it supports windows)

@mattgodbolt you can use RUSTUP_HOME and CARGO_HOME to control where the toolchains and rustup proxies are installed respectively. I'm not sure if this will solve your use-case?

[cuviper]

Or more precisely, rust-lang/rust-installer#59 defines sysconfdir here. I understand why that's not under the prefix (like /usr/etc), but it seems surprising that it doesn't at least respect DESTDIR.

cc @Keruspe

[cuviper]

Actually, I think it should use DESTDIR_PREFIX/etc/ by default, then let users choose --sysconfdir=/etc if they're sure they want it installed globally.

rustup just puts these files in $RUSTUP_HOME/toolchain/$version/etc/.

[Keruspe]

When using x.py it respects DESTDIR just like all the others (libdir, bindir, etc).
Is there a way to reproduce?

[cuviper]

$ wget https://static.rust-lang.org/dist/cargo-nightly-x86_64-unknown-linux-gnu.tar.gz
$ tar xf cargo-nightly-x86_64-unknown-linux-gnu.tar.gz 
$ ./cargo-nightly-x86_64-unknown-linux-gnu/install.sh --prefix=./usr
install: creating uninstall script at /tmp/foo/usr/lib/rustlib/uninstall.sh
install: installing component 'cargo'
install: backing up existing file at /etc/bash_completion.d/cargo
mv: cannot move '/etc/bash_completion.d/cargo' to '/etc/bash_completion.d/cargo.old': Permission denied
install: error: failed to back up /etc/bash_completion.d/cargo. see logs at '/tmp/foo/usr/lib/rustlib/install.log'

[cuviper]

@Keruspe I think it should just be like this to be prefixed by default:

diff --git a/install-template.sh b/install-template.sh
index 02070c35dcd4..fc8aa7985256 100644
--- a/install-template.sh
+++ b/install-template.sh
@@ -866,7 +866,7 @@ fi
 valopt without "" "comma-separated list of components to not install"
 valopt components "" "comma-separated list of components to install"
 flag list-components "list available components"
-valopt sysconfdir "/etc" "install system configuration files"
+valopt sysconfdir "$CFG_DESTDIR_PREFIX/etc" "install system configuration files"
 valopt bindir "$CFG_DESTDIR_PREFIX/bin" "install binaries"
 valopt libdir "$CFG_DESTDIR_PREFIX/lib" "install libraries"
 valopt mandir "$CFG_DESTDIR_PREFIX/share/man" "install man pages in PATH"

(i.e. don't assume that we really want the global /etc by default)

[Keruspe]

See rust-lang/rust-installer#64
It seems pretty standard not to respect prefix for sysconfdir, but not respecting DESTDIR was indeed a mistake