Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat(tools/opt-dist): allow local builds to specify a rustc-perf checkout #125125

Merged
merged 1 commit into from
May 14, 2024

Conversation

lovesegfault
Copy link
Contributor

This is a first step towards allowing opt-dist to work in a sandboxed /
air-gapped environment, as it allows users to bypass the ad-hoc download of
rustc-perf.

@rustbot
Copy link
Collaborator

rustbot commented May 14, 2024

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 14, 2024
@Mark-Simulacrum
Copy link
Member

@bors r+ rollup

@bors
Copy link
Contributor

bors commented May 14, 2024

📌 Commit c3c9783 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2024
@bors
Copy link
Contributor

bors commented May 14, 2024

⌛ Testing commit c3c9783 with merge 8387315...

@bors
Copy link
Contributor

bors commented May 14, 2024

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 8387315 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label May 14, 2024
@bors bors merged commit 8387315 into rust-lang:master May 14, 2024
7 checks passed
@rustbot rustbot added this to the 1.80.0 milestone May 14, 2024
@lovesegfault lovesegfault deleted the opt-dist-specify-rustc-perf branch May 14, 2024 21:05
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (8387315): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary -1.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-1.2% [-1.2%, -1.2%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -1.2% [-1.2%, -1.2%] 1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 679.638s -> 678.372s (-0.19%)
Artifact size: 316.03 MiB -> 316.02 MiB (-0.00%)

bors pushed a commit to rust-lang-ci/rust that referenced this pull request May 16, 2024
Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:

`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:

```dockerfile
ENV PERF_COMMIT 4f313ad
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
    unzip perf.zip && \
    mv rustc-perf-$PERF_COMMIT rustc-perf && \
    rm perf.zip
```

`src/tools/opt-dist/src/main.rs`

```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";

let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
    .timeout(Duration::from_secs(60 * 2))
    .connect_timeout(Duration::from_secs(60 * 2))
    .build()?;
let response = retry_action(
    || Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
    "Download rustc-perf archive",
    5,
)?;
```

This causes a few issues:

1. Maintainers need to be careful to bump PERF_COMMIT in both places
   every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
   `rustc-perf` (rust-lang#125125), but to
   figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
   dependencies are not included in the `vendor/` dir created during
   `dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
   `rustc-perf` in use, leading to staleness

Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.

This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313ad. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
bors added a commit to rust-lang-ci/rust that referenced this pull request May 16, 2024
refactor: add rustc-perf submodule to src/tools

Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:

`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:

```dockerfile
ENV PERF_COMMIT 4f313ad
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
    unzip perf.zip && \
    mv rustc-perf-$PERF_COMMIT rustc-perf && \
    rm perf.zip
```

`src/tools/opt-dist/src/main.rs`

```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";

let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
    .timeout(Duration::from_secs(60 * 2))
    .connect_timeout(Duration::from_secs(60 * 2))
    .build()?;
let response = retry_action(
    || Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
    "Download rustc-perf archive",
    5,
)?;
```

This causes a few issues:

1. Maintainers need to be careful to bump PERF_COMMIT in both places
   every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
   `rustc-perf` (rust-lang#125125), but to
   figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
   dependencies are not included in the `vendor/` dir created during
   `dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
   `rustc-perf` in use, leading to staleness

Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.

This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313ad. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
bors added a commit to rust-lang-ci/rust that referenced this pull request May 17, 2024
refactor: add rustc-perf submodule to src/tools

Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:

`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:

```dockerfile
ENV PERF_COMMIT 4f313ad
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
    unzip perf.zip && \
    mv rustc-perf-$PERF_COMMIT rustc-perf && \
    rm perf.zip
```

`src/tools/opt-dist/src/main.rs`

```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";

let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
    .timeout(Duration::from_secs(60 * 2))
    .connect_timeout(Duration::from_secs(60 * 2))
    .build()?;
let response = retry_action(
    || Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
    "Download rustc-perf archive",
    5,
)?;
```

This causes a few issues:

1. Maintainers need to be careful to bump PERF_COMMIT in both places
   every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
   `rustc-perf` (rust-lang#125125), but to
   figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
   dependencies are not included in the `vendor/` dir created during
   `dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
   `rustc-perf` in use, leading to staleness

Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.

This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313ad. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
lovesegfault added a commit to lovesegfault/rust that referenced this pull request May 17, 2024
Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:

`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:

```dockerfile
ENV PERF_COMMIT 4f313ad
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
    unzip perf.zip && \
    mv rustc-perf-$PERF_COMMIT rustc-perf && \
    rm perf.zip
```

`src/tools/opt-dist/src/main.rs`

```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";

let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
    .timeout(Duration::from_secs(60 * 2))
    .connect_timeout(Duration::from_secs(60 * 2))
    .build()?;
let response = retry_action(
    || Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
    "Download rustc-perf archive",
    5,
)?;
```

This causes a few issues:

1. Maintainers need to be careful to bump PERF_COMMIT in both places
   every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
   `rustc-perf` (rust-lang#125125), but to
   figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
   dependencies are not included in the `vendor/` dir created during
   `dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
   `rustc-perf` in use, leading to staleness

Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.

This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313ad. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
lovesegfault added a commit to lovesegfault/rust that referenced this pull request May 20, 2024
Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:

`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:

```dockerfile
ENV PERF_COMMIT 4f313ad
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
    unzip perf.zip && \
    mv rustc-perf-$PERF_COMMIT rustc-perf && \
    rm perf.zip
```

`src/tools/opt-dist/src/main.rs`

```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";

let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
    .timeout(Duration::from_secs(60 * 2))
    .connect_timeout(Duration::from_secs(60 * 2))
    .build()?;
let response = retry_action(
    || Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
    "Download rustc-perf archive",
    5,
)?;
```

This causes a few issues:

1. Maintainers need to be careful to bump PERF_COMMIT in both places
   every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
   `rustc-perf` (rust-lang#125125), but to
   figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
   dependencies are not included in the `vendor/` dir created during
   `dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
   `rustc-perf` in use, leading to staleness

Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.

This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313ad. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
bors added a commit to rust-lang-ci/rust that referenced this pull request May 20, 2024
…k-Simulacrum

refactor: add rustc-perf submodule to src/tools

Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:

`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:

```dockerfile
ENV PERF_COMMIT 4f313ad
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
    unzip perf.zip && \
    mv rustc-perf-$PERF_COMMIT rustc-perf && \
    rm perf.zip
```

`src/tools/opt-dist/src/main.rs`

```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";

let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
    .timeout(Duration::from_secs(60 * 2))
    .connect_timeout(Duration::from_secs(60 * 2))
    .build()?;
let response = retry_action(
    || Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
    "Download rustc-perf archive",
    5,
)?;
```

This causes a few issues:

1. Maintainers need to be careful to bump PERF_COMMIT in both places
   every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
   `rustc-perf` (rust-lang#125125), but to
   figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
   dependencies are not included in the `vendor/` dir created during
   `dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
   `rustc-perf` in use, leading to staleness

Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.

This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313ad. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
weihanglo added a commit to weihanglo/rust that referenced this pull request May 23, 2024
These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

These crates are the default set of packages required by opt-dist
to correctly work, hence for people wanting to build a production grade
of rustc in an sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request May 24, 2024
bootstrap: vendor crates required by opt-dist to collect profiles

These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
flip1995 pushed a commit to flip1995/rust-clippy that referenced this pull request May 24, 2024
refactor: add rustc-perf submodule to src/tools

Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:

`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:

```dockerfile
ENV PERF_COMMIT 4f313add609f43e928e98132358e8426ed3969ae
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
    unzip perf.zip && \
    mv rustc-perf-$PERF_COMMIT rustc-perf && \
    rm perf.zip
```

`src/tools/opt-dist/src/main.rs`

```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";

let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
    .timeout(Duration::from_secs(60 * 2))
    .connect_timeout(Duration::from_secs(60 * 2))
    .build()?;
let response = retry_action(
    || Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
    "Download rustc-perf archive",
    5,
)?;
```

This causes a few issues:

1. Maintainers need to be careful to bump PERF_COMMIT in both places
   every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
   `rustc-perf` (rust-lang/rust#125125), but to
   figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
   dependencies are not included in the `vendor/` dir created during
   `dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
   `rustc-perf` in use, leading to staleness

Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.

This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313add609f43e928e98132358e8426ed3969ae. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
weihanglo added a commit to weihanglo/rust that referenced this pull request May 31, 2024
These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

These crates are the default set of packages required by opt-dist
to correctly work, hence for people wanting to build a production grade
of rustc in an sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
celinval pushed a commit to celinval/rust-dev that referenced this pull request Jun 4, 2024
Update Rust toolchain from nightly-2024-05-14 to nightly-2024-05-15
without any other source changes.
This is an automatically generated pull request. If any of the CI checks
fail, manual intervention is required. In such a case, review the
changes at https://github.com/rust-lang/rust from
rust-lang@ab14f94
up to
rust-lang@8387315.
The log for this commit range is:
rust-lang@8387315ab3 Auto merge of
rust-lang#125125 - lovesegfault:opt-dist-specify-rustc-perf, r=Mark-Simulacrum
rust-lang@c3c9783de2
feat(tools/opt-dist): allow local builds to specify a rustc-perf
checkout
rust-lang@ac385a5af6 Auto merge of
rust-lang#125120 - compiler-errors:rollup-mnjybwv, r=compiler-errors
rust-lang@31016d5879 Rollup merge of
rust-lang#125118 - GuillaumeGomez:cleanup-run-make, r=jieyouxu
rust-lang@d59f430eec Rollup merge of
rust-lang#125100 - compiler-errors:faster, r=nnethercote
rust-lang@712e7c37f7 Rollup merge of
rust-lang#125088 - compiler-errors:uplift-alias-ty, r=lcnr
rust-lang@8c64acdbdc Rollup merge of
rust-lang#125080 - bvanjoi:fix-124946, r=nnethercote
rust-lang@844c7e826e Rollup merge of
rust-lang#125047 - Oneirical:test5, r=jieyouxu
rust-lang@0458d8a53b Rollup merge of
rust-lang#124844 - compiler-errors:shadow-probe, r=lcnr
rust-lang@36287830a2 Rollup merge of
rust-lang#119838 - joshtriplett:style-guide-binop-indent, r=compiler-errors
rust-lang@ade33b02f2 only find segs chain
for missing methods when no available candidates
rust-lang@bdfd941f4d Auto merge of
rust-lang#123816 - tgross35:f16-f128-mangling, r=michaelwoerister
rust-lang@f97d915173 Use new utility
functions/methods in run-make tests
rust-lang@792a9bdd4b Enable v0 mangling
tests and add checks for `f16`/`f128`
rust-lang@809b84edba Add v0 symbol
mangling for `f16` and `f128`
rust-lang@31026b7fe3 Auto merge of
rust-lang#125023 - morr0ne:linux-none-target, r=Nilstrieb
rust-lang@68407f9049 fix typo in
x86_64-unknown-linux-none docs
rust-lang@c45e831d8f Auto merge of
rust-lang#124228 - compiler-errors:lint-overcaptures, r=oli-obk
rust-lang@58426f4a5b Auto merge of
rust-lang#125026 - Oneirical:clink-tests, r=jieyouxu
rust-lang@e098eb14ae Wording improvement
rust-lang@e2d9c0d938 Fix missing word
rust-lang@57c32a193f style-guide: When
breaking binops handle multi-line first operand better
rust-lang@dbd2ca6478 Use a proper probe
for shadowing impl
rust-lang@052de1da4f And finally add
tests
rust-lang@1529c661e4 Warn against
redundant use<...>
rust-lang@f3fb727b08 Don't suggest using
use<> syntax to capture APITs
rust-lang@6afe1352d9 Suggest adding use<>
syntax
rust-lang@554becc180 Add some commenting
rust-lang@d57e57ca1f Implement initial
IMPL_TRAIT_OVERCAPTURES lint
rust-lang@8f97a2588c Add test to make
sure suggestions are still quick
rust-lang@fba5f44bd8 Auto merge of
rust-lang#125098 - jhpratt:rollup-2qm4gga, r=jhpratt
rust-lang@45b50d303c lto function,
static_library call, rename
rust-lang@9f8cdb286e Remove to_term
rust-lang@1ad28a6f53 Uplift AliasTy
rust-lang@812f89728a fix fmt
rust-lang@2e4c90c3f7 Don't do
post-method-probe error reporting steps if we're in a suggestion
rust-lang@32d74f1800 Rollup merge of
rust-lang#125090 - erickt:bump-fuchsia, r=tmandry
rust-lang@209703af85 Rollup merge of
rust-lang#125072 - Darksonn:pin-dyn-dispatch-sound, r=jhpratt
rust-lang@18d9c039bb Rollup merge of
rust-lang#124997 - gurry:124848-ice-should-be-sized, r=Nadrieril
rust-lang@74a78af0e2 Rollup merge of
rust-lang#116675 - joshlf:patch-10, r=scottmcm
rust-lang@9105c57b7f Auto merge of
rust-lang#124256 - nnethercote:rm-NtIdent-NtLifetime, r=petrochenkov
rust-lang@34582118af Auto merge of
rust-lang#125076 - compiler-errors:alias-term, r=lcnr
rust-lang@95e519ecbf Remove `NtIdent` and
`NtLifetime`.
rust-lang@fa84018c2e Apply nits
rust-lang@58ee9192e0 Migrate fuchsia docs
from `pm` to `ffx`
rust-lang@293b5cb1ca [ptr] Document
maximum allocation size
rust-lang@3bcdf3058e split out AliasTy ->
AliasTerm
rust-lang@b3a78c1d09 Add test for dynamic
dispatch + Pin::new soundness
rust-lang@9a63a42cb7 Remove a `Span` from
`TokenKind::Interpolated`.
rust-lang@71fd2cf5b4 fix function call
and import
rust-lang@a1b5ea0cc2 make tidy happy
rust-lang@f2de5fb2ae rewrite issue-14500
to rmake
rust-lang@a6f237ca85 docs: fix typo in
platform-support docs
rust-lang@923cdb35aa test: Add assembly
tests for x86_64-unknown-linux-none target
rust-lang@10c358f111 Make tidy happy
rust-lang@e37d2989c1 remove trailing
whitespace
rust-lang@a2e7e79a13 Port
c-link-to-rust-va-list-fn to Rust
rust-lang@9cf080099d docs: Document
x86_64-unknown-linux-none target
rust-lang@a3ef01b1fc Add
x86_64-unknown-linux-none target
rust-lang@fb619ec208 FIx ICE while
casting a type with error

Co-authored-by: tautschnig <1144736+tautschnig@users.noreply.github.com>
weihanglo added a commit to weihanglo/rust that referenced this pull request Jun 5, 2024
These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

These crates are the default set of packages required by opt-dist
to correctly work, hence for people wanting to build a production grade
of rustc in an sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
weihanglo added a commit to weihanglo/rust that referenced this pull request Jun 9, 2024
These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

These crates are the default set of packages required by opt-dist
to correctly work, hence for people wanting to build a production grade
of rustc in an sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
jieyouxu added a commit to jieyouxu/rust that referenced this pull request Jun 9, 2024
…Simulacrum

bootstrap: vendor crates required by opt-dist to collect profiles

These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
jieyouxu added a commit to jieyouxu/rust that referenced this pull request Jun 9, 2024
…Simulacrum

bootstrap: vendor crates required by opt-dist to collect profiles

These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jun 9, 2024
Rollup merge of rust-lang#125465 - weihanglo:opt-dist-vendor, r=Mark-Simulacrum

bootstrap: vendor crates required by opt-dist to collect profiles

These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
lcnr pushed a commit to lcnr/rust that referenced this pull request Jun 12, 2024
These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

These crates are the default set of packages required by opt-dist
to correctly work, hence for people wanting to build a production grade
of rustc in an sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: rust-lang#125166 (comment)

Previous efforts on making:

* rust-lang#125125
* rust-lang#125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: https://github.com/rust-lang/rustc-perf/blob/4f313add609f43e928e98132358e8426ed3969ae/collector/src/compile/benchmark/mod.rs#L164-L173
[^2]: https://github.com/rust-lang/rust/blob/606afbb617a2949a4e35c4b0258ff94c980b9451/src/tools/opt-dist/src/tests.rs#L62-L77
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants