clang -fuse-ld=lld --target= prefers target-prefixed lld binaries over non-prefixed lld in its own directory

[zmodem]

clang's --target flag affects which lld binary it chooses. That seems reasonable, but it can lead to surprising behaviour if there are multiple toolchains available.

Let's pretend we have some old crosscompiling toolchain in our PATH:

$ mkdir -p /tmp/somepath/bin
$ touch /tmp/somepath/bin/x86_64-linux-ld.lld
$ chmod +x /tmp/somepath/bin/x86_64-linux-ld.lld
$ export PATH=$PATH:/tmp/somepath/bin

clang will normally prefer the lld installed right next to itself:

$ touch a.o
$ build/bin/clang -### -fuse-ld=lld a.o
[...]
build/bin/ld.lld [...]

However, if we pass a --target flag, it prefers the lld version that has that target prefix even if this is not in the clang directory:

$ build/bin/clang -### --target=x86_64-linux -fuse-ld=lld a.o
[...]
/tmp/somepath/bin/x86_64-linux-ld.lld [...]

That seems potentially surprising!

It seems there are two criteria at play here:

1. clang will prefer an lld binary in its own directory over binaries on PATH
2. clang will prefer an lld binary with the target prefix over one without

It seems that 2) currently takes precedence. Maybe it should be the other way around?

[llvmbot]

@llvm/issue-subscribers-clang-driver

Author: Hans Wennborg (zmodem)

clang's `--target` flag affects which `lld` binary it chooses. That seems reasonable, but it can lead to surprising behaviour if there are multiple toolchains available.

Let's pretend we have some old crosscompiling toolchain in our PATH:

$ mkdir -p /tmp/somepath/bin
$ touch /tmp/somepath/bin/x86_64-linux-ld.lld
$ chmod +x /tmp/somepath/bin/x86_64-linux-ld.lld
$ export PATH=$PATH:/tmp/somepath/bin

clang will normally prefer the lld installed right next to itself:

$ touch a.o
$ build/bin/clang -### -fuse-ld=lld a.o
[...]
build/bin/ld.lld [...]

However, if we pass a --target flag, it prefers the lld version that has that target prefix even if this is not in the clang directory:

$ build/bin/clang -### --target=x86_64-linux -fuse-ld=lld a.o
[...]
/tmp/somepath/bin/x86_64-linux-ld.lld [...]

That seems potentially surprising!

It seems there are two criteria at play here:

1. clang will prefer an lld binary in its own directory over binaries on PATH
2. clang will prefer an lld binary with the target prefix over one without

It seems that 2) currently takes precedence. Maybe it should be the other way around?

[zmodem]

+cc @nico @inglorion

[nico]

Maybe d6efc98 changed things to the current state? @DavidSpickett

The relevant code is ToolChain::GetLinkerPath which calls Driver::GetProgramPath.

[nico]

That revision is a reland of 028571d which points at https://llvm.org/PR45693 / #45038 as motivation for the current behavior.

[zmodem]

Aha, and that points to an RFC about exactly this: https://discourse.llvm.org/t/rfc-changing-tool-search-path-priority/55341

[DavidSpickett]

It seems there are two criteria at play here:

1. clang will prefer an lld binary in its own directory over binaries on PATH
2. clang will prefer an lld binary with the target prefix over one without

It seems that 2) currently takes precedence.

Yes, Clang currently starts with the most specific name and looks for that in its install directory and then on PATH. If that fails, it looks for the next name in both places and so on.

Maybe it should be the other way around?

At the time, my opinion was that for the use case in the bug report, 2) then 1) made the most sense. Though I could see how you would disagree if file system "distance" is the first criteria in your mental model of this.

(and to be honest, I can't remember what I thought it would be, prior to seeing that bug report)

In the situation where you need a cross compiler gcc, it was finding gcc first. Which was never going to work. If gcc was a cross compiler by default like clang, then 1) 2) order would have also worked. Which may apply for lld too, but probably we would want to keep the search logic the same for all tools.

Not sure how to proceed here.

One way might be to look at the workaround that the original bug report would have to have done (a couple were suggested at the time), vs. what your use case(s) would have to do.

As I am clearly too biased to be saying which strategy makes more sense intuitively.

[zmodem]

Not sure how to proceed here.

Yeah, I'm not sure either. We worked around this on our end, but it was definitely a surprise, so I figured it made sense to raise it.