Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add two-stage build support in source-build #112414

Closed
wants to merge 5 commits into from
Closed

Conversation

am11
Copy link
Member

@am11 am11 commented Feb 11, 2025

The source-build for community platforms are also running into LKG issue, for which we added TwoStage build support filipnavara/dotnet-riscv#4. This PR exposes two-stage build properties (StageOneBuild and StageTwoBuild) for source-build so we can build the product end-to-end for community platforms.

Also, Microsoft.NETCore.App.Runtime.CoreCLR tries to run crossgen2, which runs into invalid binary. Disabling it for StageTwoBuild for now as it's also disabled for FreeBSD etc.

@jkoritzinsky
Copy link
Member

@am11, instead of hooking up the multistage build to SB/VMR like this, we'd like it to have the following UX:

In the eng/build.sh and eng/build.ps1 scripts, provide the following arguments:

  • --build-bootstrap: Builds the Stage1 and Stage2 builds, puts the artifacts into an artifacts/bootstrap directory.
  • --use-bootstrap: Builds the regular build, using artifacts in artifacts/bootstrap.
  • --build-with-bootstrap: Run build.sh with --build-bootstrap, then run the build as though --use-bootstrap was passed.

Then, in the VMR, we can add support for a /p:DotNetRuntimeBootstrap=true property that the runtime.proj in the VMR will forward to the runtime build. In eng/DotNetBuild.props here, that logic would add --build-with-bootstrap to the inner build argument when DotNetRuntimeBootstrap is true.

Can you update this PR (and the one in dotnet/sdk) to reflect this UX?

@am11
Copy link
Member Author

am11 commented Feb 12, 2025

The approach makes lot of sense but I am not sure if it is the right "granularity". Bootstraping runtime alone and manipulating nuget source feed paths manually is not very smooth.

If we bootstrap VMR (dotnet/dotnet) and then use that bootstrapped SDK to build the product again, that makes more sense. i.e. revive https://github.com/dotnet/source-build/blob/main/Documentation/bootstrapping-guidelines.md and cover --bootstrap in top-level script such that it's passed to runtime, msbuild and other repos which support bootstrapping.

With current state of this PR, user can simulate bootstrapping by rerunning VMR build.sh and pointing to the previously built SDK by two-stage process.

TL;DR we should definitely think about bootstrapping more holistically. Meanwhile, these two tiny PRs in rutnime and sdk are unblocking community platform source-build which is stuck for months. The bootstrap solution can revert this stop-gap solution once it arrives (which may happen in weeks, may take months or longer..).

@am11
Copy link
Member Author

am11 commented Feb 14, 2025

@jkoritzinsky do you think we can take these two (total of 30 LOC) PRs to fix the community platforms source-build and later tackle the bootstrap user-story? (which will take a lot more work than 30 lines because that solution doesn't exist!)

cc @jkotas this is a regression fix for community platform source-build. We cannot build net10.0 preview 1 without it.

@am11 am11 requested review from akoeplinger and jkotas February 14, 2025 06:38
@jkotas
Copy link
Member

jkotas commented Feb 14, 2025

I do not have much else to say than what I have said for the previous PR in this series #105004 (comment) . I am sympathetic to the concerns of the build system maintainers about where this series is going.

cc @jkotas this is a regression fix for community platform source-build. We cannot build net10.0 preview 1 without it.

I am sorry about that There are a lot of changes going into the build system as we are preparing to switch our official builds over to use VMR. It is likely that source builds for community targets will need local patches throughout .NET 10 previews.

--build-bootstrap: Builds the Stage1 and Stage2 builds, puts the artifacts into an artifacts/bootstrap directory.
--use-bootstrap: Builds the regular build, using artifacts in artifacts/bootstrap

Current Stage2 build is closer to --use-bootstrap build. I do not think --build-bootstrap should be implemented as Stage1 + Stage2 builds as they exist today.

@am11
Copy link
Member Author

am11 commented Feb 14, 2025

@jkotas, thank you for the clarification. This is just a short term stop-gap solution to propagate stage options from VMR to runtime. Granted it isn't pretty but it's a tiny hook to use what runtime's CI exercises.

https://github.com/dotnet/source-build/ repo has an existing concept of bootstrapping for many years. SB and distro maintainers are familiar with it. They have --with-packages and --with-sdk options for bootstrapping. e.g. I was reading @ViktorHofer's recent comments dotnet/source-build#4891 (comment) and tried it out with cross-community build: https://github.com/am11/dotnet-riscv/actions/runs/13313853374/workflow#L26 (and second one is on line 47 using --with-packages), but it's look for reference packages which are missing (I haven't got time to figure out where to collect it).

I think, in long run, it would be a good idea if we take @jkoritzinsky's proposal into consideration in light of those existing facilities (preferably without introducing new options to top-level SB).

@am11 am11 closed this Feb 17, 2025
@am11 am11 deleted the patch-30 branch February 17, 2025 04:56
@am11 am11 mentioned this pull request Feb 17, 2025
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
area-Infrastructure community-contribution Indicates that the PR has been added by a community member
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants