Skip to content

Bootstrap cannot remove host symlink on Windows if created from WSL2 #112544

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
ChrisDenton opened this issue Jun 12, 2023 · 1 comment · Fixed by #129187
Open

Bootstrap cannot remove host symlink on Windows if created from WSL2 #112544

ChrisDenton opened this issue Jun 12, 2023 · 1 comment · Fixed by #129187
Labels
C-bug Category: This is a bug. O-windows Operating system: Windows T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@ChrisDenton
Copy link
Member

If I use ./x in WSL2 it will create a host symlink in the build directory. If I then switch to the Windows host using the same build directory I get this error:

thread 'main' panicked at 'fs::remove_dir(&host) failed with The directory name is invalid. (os error 267)', lib.rs:516:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The simplest solution here would be to detect ERROR_DIRECTORY (aka os error 267) and fallback to fs::remove_file.

@ChrisDenton ChrisDenton added O-windows Operating system: Windows T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) C-bug Category: This is a bug. labels Jun 12, 2023
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 17, 2024
…nks, r=<try>

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`. Because I wasn't super sure about `std::fs::remove_dir_all`'s behavior and to catch `std::fs::remove_dir_all`'s behavioral changes here on forward, I added a collection of tests that checks if our expectation of the behavior of `std::fs::remove_dir_all` and its underlying Unix syscalls and Win32 APIs matches with its actual behavior.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows:

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 17, 2024
…nks, r=<try>

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 18, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 18, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 18, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 18, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 20, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 20, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
tgross35 added a commit to tgross35/rust that referenced this issue Aug 21, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
tgross35 added a commit to tgross35/rust that referenced this issue Aug 21, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 22, 2024
…links, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
@bors bors closed this as completed in 9fd8a2c Aug 22, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Aug 22, 2024
Rollup merge of rust-lang#129187 - jieyouxu:squeaky-clean-windows-symlinks, r=Kobzol

bootstrap: fix clean's remove_dir_all implementation

It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.

This is a blocker for compiletest's own `rm_rf` implementation rust-lang#129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port rust-lang#128562 that heavily exercises symlinks (I was reviewing rust-lang#128562 and testing it on native Windows which is how I found out).

I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened rust-lang#129188):

```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
  left: "E:\\tmp"
 right: "C:\\tmp"
```

Fixes rust-lang#112544 (because now we handle Windows symlinks properly).

try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
@jieyouxu
Copy link
Member

Re-opening due to revert #129413.

@jieyouxu jieyouxu reopened this Aug 22, 2024
@jieyouxu jieyouxu self-assigned this Aug 22, 2024
@jieyouxu jieyouxu removed their assignment Oct 9, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
C-bug Category: This is a bug. O-windows Operating system: Windows T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants