PublishTrimmed should imply self-contained

[eerhardt]

Doing the following steps:

• dotnet new console
• dotnet publish -p:PublishTrimmed=true

Results in the following error:

C:\Users\eerhardt\.nuget\packages\microsoft.net.illink.tasks\8.0.100-1.22619.1\build\Microsoft.NET.ILLink.targets(196,5): error NETSDK1102: Optimizing assemblies for size is not supported for
the selected publish configuration. Please ensure that you are publishing a self-contained app. [C:\DotNetTest\Net8Console\Net8Console.csproj]

However, doing

• dotnet publish -p:PublishAot=true
• -or-
• dotnet publish -p:PublishSingleFile=true

Just works without more command line args.

We should make PublishTrimmed=true imply SelfContained=true, just like PublishAot=true and PublishSingleFile=true does. That way users don't need to pass extra args on the command line, just to make this work.

If in the future we want to support a mode where PublishTrimmed=true, but SelfContained=false, those customers can explicitly --self-contained false. This is consistent with the current PublishSingleFile=true behavior. By default it is self-contained. And you can opt into Framework Dependent Deployment with dotnet publish -p:PublishSingleFile=true --self-contained false.

cc @vitek-karas @sbomer @agocke

[ghost]

@dotnet/linker-contrib a new issue has been filed in the ILLink area, please triage

[agocke]

Surprised that dotnet publish -p:PublishSingleFile=true is self-contained by default. But I don't dislike the behavior. And agree with the proposal about PublishTrimmed.

The only thing I don't like is that these properties affect dotnet build as well. In fact, I don't like self-contained dotnet build at all. I think that should be a publish-only gesture.

[sbomer]

I think this is happening because of the default RID selection. PublishReadyToRun/PublishSingleFile/PublishAot use the current RID, and having a RID currently implies self-contained.

sdk/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.RuntimeIdentifierInference.targets

Lines 79 to 92 in 201d719

	<UseCurrentRuntimeIdentifier Condition="
	'$(RuntimeIdentifier)' == '' and
	'$(_IsExecutable)' == 'true' and '$(IsTestProject)' != 'true' and
	'$(IsRidAgnostic)' != 'true' and
	(
	'$(SelfContained)' == 'true' or
	('$(_IsPublishing)' == 'true' and
	(
	'$(PublishReadyToRun)' == 'true' or
	'$(PublishSingleFile)' == 'true' or
	'$(PublishAot)' == 'true'
	)
	)
	)">true</UseCurrentRuntimeIdentifier>

But @nagilson is working on a breaking change where RID no longer implies self-contained: #30038.

I expect that after this change, PublishAot will have an error similar to the current one for PublishTrimmed, and the PublishSingleFile default will become framework-dependent. The latter might violate user expectations, but at least will be consistent with the new defaults.

[sbomer]

@marcpopMSFT IIRC you set up the auto-tagging of @dotnet/linker-contrib based on the area labels. The team name changed to @dotnet/illink-contrib. Would you be able to update the bot config?

[marcpopMSFT]

@sbomer see #30152. Feel free to update this in the future if there are changes needed.

[richlander]

Good topic! We're trying to provide coherent behaviors for some things that are similar and others less so. It's sometimes easier when we group the options such that the answer pops out. Let's see if that happens here.

I'm going to use MSBuild syntax, where all properties are intended as true or otherwise have coherent value.

Always framework-dependent:

• [nothing]
• RuntimeIdentifier

Supports both:

• PublishReadyToRun
• PublishSingleFile

Could support both, but only supports self-contained:

• PublishTrimmed

Always self-contained:

• PublishAot
• PublishSelfContained
• SelfContained

There are two ways we could go with this:

• Correctness
• Convenience

There are four groups. The pragmatic choice is make the first two groups fx-dependent by default and the second two are self-contained.

Another observation is that PublishTrimmed is likely not used that often on its own. Instead, we could do the following:

• Require users to specifying self-contained if they use PublishTrimmed
• Default PublishSelfContained to infer PublishTrimmed and require users to disable trimming if they don't want it. We could also infer PublishReadyToRun in the same way.

That approach would have multiple benefits:

• Most of the options would be neutral to the output type (which is good).
• We'd have an option for producing self-contained apps that is good. Most people would agree that the current default for self-contained apps is not polished/optimal.
• Single file would remain a separate concern. The thinking to date is that the various forms of single file should be opt-in since there are more compat considerations with single file variants than with self-contained.

This approach seems to balance CLI consistency with convenience while also most often providing users with a set of artifacts that we would say are "good".

[sbomer]

Default PublishSelfContained to infer PublishTrimmed and require users to disable trimming if they don't want it. We could also infer PublishReadyToRun in the same way.

It's worth noting that PublishTrimmed and PublishReadyToRun can be somewhat at odds, depending on the goals - trimming will remove R2R info and produce the smallest output, while R2R adds back some size in exchange for startup perf.

Single file would remain a separate concern. The thinking to date is that the various forms of single file should be opt-in since there are more compat considerations with single file variants than with self-contained.

It seems like the same reasoning would apply to PublishTrimmed as well - there are compat concerns if we implicitly set PublishTrimmed for PublishSelfContained.

[agocke]

@richlander Based on my comment above, I want to retire SelfContained as it is today. Specifically, I think if you want a self-contained output you should have to use dotnet publish.

[agocke]

A couple more things, actually.

Default PublishSelfContained to infer PublishTrimmed and require users to disable trimming if they don't want it. We could also infer PublishReadyToRun in the same way.

I'm not sure I understand what you're suggesting here. You mean that if PublishSelfContained is true, the default for PublishTrimmed is also true? Or the other way around?

Single file would remain a separate concern. The thinking to date is that the various forms of single file should be opt-in since there are more compat considerations with single file variants than with self-contained.

The compat between self-contained single file and plain self-contained is getting pretty good at this point. It's pretty much just Assembly.Location now.

[richlander]

That's a good point @sbomer. Let's go with that.

I'm thinking about this more. I was trying to get at something with that categorization in my previous post, but I didn't quite know what. I do now.

Let's try again.

App characteristics (don't auto-switch from FDD -> SCD):

• PublishReadyToRun
• PublishSingleFile
• PublishTrimmed
• RuntimeIdentifier

App kind (these auto switch to SCD):

• PublishNativeAot
• PublishSelfContained
• SelfContained

This feels like a more useful categorization, but I still don't find it particularly satisfying. For example, PublishSingleFile feels like it should be an app kind and PublishTrimmed only supports self-contained apps currently.

In the ideal case, users set the minimum values in their project file, like setting PublishSelfContained=true. If they want to try out trimming or single file from the CLI, they will always get the desired result.

I think this conversation comes down to specializing a default project file (like what templates produce) via the CLI. I think a solution like this would help: #26249 (comment). It's basically the same thing as specifying the same values in a project file.

@agocke -- SelfContained ... I'm a little torn on that one. We need to decide once and for all that we're going to stay with publish forever. If we agree to that, then I'm happy to go along with your suggestion. Related: #26247.

@agocke -- I meant that PublishSelfContained could infer PublishTrimmed. I had proposed single file as a default in the past. @jkotas considered that too much of a compatibility risk for opt-out.

Upon further thinking, I think we should go with the categorization I have above and then rely on good project file and CLI gestures to make it easy to adopt the configuration you want. We just added PublishSelfContained and the bad PM for that feature hasn't published a blog post on that yet. I think we don't have enough customer feedback to suggest we need something more dramatic. I'd rather see us make the CLI easier to use to specify defaults for an environment.

[richlander]

@baronfel @dsplaisted

[nagilson]

Thanks everyone for engaging here + internally 😄

@richlander Should we still take this change: #30038 if it does not have PublishAot giving SelfContained by default? Today is the last day for it to get merged (technically Monday at 3 PM...). It sounds like we are OK with this change breaking PublishTrimmed, PublishSingleFile, and PublishReadyToRun such that old usages of these will now have to add SelfContained.

If we have a finalized decision on this, we should close/edit this issue since we've seem to have decided against it, and make one specifically for having PublishAot -> SelfContained, as well as one for PublishSelfContained.

[richlander]

Right. I'm suggesting that PublishAot and PublishSelfContained should infer SelfContained.

I think all the other cases should not infer a SelfContained value. That breaks our model, IMO, and is impossible to explain w/o giving folks a decoder ring. Sometimes what feels like a productivity killer (like our PublishTrimmed example) is overshadowed by an inconsistent model.

[richlander]

Up-leveling a bit. We should be dissatisfied with experiences that are inconvenient. However, addressing those at the lowest levels (like with these properties) is just one option. We should ensure we have all the options on the table for how to address UX. I'm proposing we handle this UX concerns (like PublishTrimmed being annoying) at a higher level. It's really only annoying for the CLI scenario.

[sbomer]

If we intend to handle UX concerns at a higher level (which as I understand means through project templates, or some other way to "bake in" the chosen options), would it make sense to treat PublishAot the same as PublishTrimmed? That way it becomes very clear whether the output is SCD or FDD, based on the presence/absence of PublishSelfContained, and PublishAot is no longer the odd one out.

From the internal discussion it also sounds like a FDD PublishAot deployment is possible in theory, even if it's unlikely we would add this option.

Then we can explain everything by saying that SCD is orthogonal to the other publish options. To use your terminology, SCD is an "app kind" and everything else is an "app characteristic" - where not all combinations are supported, but nothing is implicitly set (so no decoder ring is required).

[richlander]

I don't think PublishAot and PublishTrimmed are very similar. We could enable PublishTrimmed for FDD if we wanted to. We just haven't. It also means we're one motivating customer request from doing so.

Another middling option is to expose another set of CLI flags, like --trim-scd or similar.

[jkotas]

We could enable PublishTrimmed for FDD if we wanted to. We just haven't.

The same can be said about PublishAot. (FDD was the default deployment option for .NET Native for UWP.)

[richlander]

You are right! I forgot about that. However (and correct me if I'm wrong), it seems more likely that we'll enable trimming for FDD than Native Aot for FDD. Or, do we just need to wait on a similar "one motiving scenario"?

[jkotas]

I agree.

[richlander]

That's a bit ambiguous. There were two things you could agree with. I assume you were agreeing that trimming for FDD was more likely.

[jkotas]

I assume you were agreeing that trimming for FDD was more likely.

Yep.

[richlander]

Here's a concrete proposal, since we still have time on our side.

• Adopt a more neutral approach with these properties (only PublishNativeAOT and *SelfContained imply SDC).
• Agree that project files have sufficiently descriptive means to communicate the desired outcome efficiently and effectively.
• Switch focus to improving the CLI to make it similarly efficient and effective.

I make no secret that I think this approach would be a good solution for this specific topic area and also have broader utility for even more common scenarios.

[vitek-karas]

As stated above there's a conflict between consistency and "ease of use". The above discussion makes it clear that we're proposing sacrifying "easy of use on CLI" and want consistent CLI options. (Personally I'm split on this, but if most people think that's the right approach, let's do it).
This means that for example adding trimming in CLI will be somewhat cumbersome (need to also specify SCD).

With that, I don't see why we would special case PublishNativeAOT as the only which implies SCD. On the scale of "basic" to "advanced" uses, AOT is probably the most advanced right now, so we could argue that making it easy to use from CLI is low priority.

Another way to look at it - if we think we can solve the UX for trimming outside of CLI, I don't see why we could not solve it the same way for AOT as well. And then keep the CLI 100% consistent.

As already mentioned, we could introduce CLI-only aliases which solve the UX:
We're already introducing dotnet new api -aot and thus it would make sense to have dotnet publish -aot which will "do the right thing" (and that could mean different things for different TFMs if we really needed to). Basically, decouple the MSBuild properties (very explicit and orthogonal/independent) from use-of-use CLI options (targeted at the most frequent scenarios).

[eerhardt]

I don't think making a breaking change for PublishSingleFile to default to framework-dependent is correct. Logically, I want my app in a single file. To me, making the default behavior being "the whole app in a single file" is the expected behavior.

For PublishTrimmed, even if/when we support FDD with PublishTrimmed, the fact that we've only had self-contained for 4, going on 5, versions now should tell us what the logical default is. If the scenario isn't important enough to implement, it isn't important enough to be considered the "default".

[sbomer]

I don't think making a breaking change for PublishSingleFile to default to framework-dependent is correct

I agree that this is one of the biggest downsides to the breaking change (RID no longer implying self-contained). Same applies to vanilla "dotnet publish -r <RID>".

However, I would find it really unexpected if "dotnet publish" produced one kind (SCD/FDD) of deployment, but "dotnet publish -p:PublishSingleFile=true" produced a different kind. So I'll throw another idea into the mix: what if we made PublishSelfContained true by default? It would keep the command-line options independent, and shifts where the breaking change is.

Pros:

• Keeps RID independent of SelfContained
• Keeps the publish optimizations independent of SCD/FDD
• Aligns with what most users will want for PublishSingleFile defaults
• Aligns with currently supported PublishTrimmed/PublishAot mode (SCD only)
• Doesn't change the output of "dotnet publish -r <RID>" (SCD)
• Doesn't change the output of "dotnet publish -p:PublishSingleFile=true" (SCD)
• Doesn't change the output of "dotnet publish -p:PublishReadyToRun=true" (SCD)

Cons:

• Vanilla "dotnet publish" output changes (was FDD, will be SCD)

I'm not sure how significant the "dotnet publish" break is. Personally I only ever do a RID-specific publish anyway (and almost always want it to be self-contained), but maybe my usage is not representative.

[nagilson]

PublishSelfContained by default is an interesting idea. Another idea per @marcpopMSFT is that we could infer SelfContained by default when using these properties, BUT have a warning like the old SelfContained warning,

WARNING: NETSDK11??: One of '--self-contained' or '--no-self-contained' options are recommended when `Publish*` is used.

Thoughts @richlander ? I think this is a good compromise with UX but also on the orthogonal properties perspective.

[richlander]

I see significant diversity of opinion for FDD trimming on this thread on feasibility, value, and likelihood.

I'd like to offer the options that Sven and I offered. Can we get behind those and choose one? FWIW, I'm happy with either so not specifically trying to push the one I suggested. It's all just tradeoffs.

[eerhardt]

@richlander - can you explain why you are cutting the "Andy and Vitek" option from consideration?

I see significant diversity of opinion for FDD trimming on this thread on feasibility, value, and likelihood.

My opinion is that even if FDD trimming was feasible, valuable, and likely, I would still want PublishAot, PublishTrimmed, and PublishSingleFile to be SCD by default.

[richlander]

I was acting on Jan's feedback, which is that we shouldn't be optimizing for the CLI but for project files for setting these properties. I agree with that. If we're talking about project files, then a consistent model trumps reducing properties.

[agocke]

I'm fine with going forward with FDD-by-default.

[richlander]

BTW: A lot of this is my fault. I pushed implying SCD for some of the Publish* settings in .NET 7. It was clearly not well thought through.

[richlander]

I propose we make that change in Preview 2 and then course correct as needed. Fair?

That means all the Publish* settings will be FDD by default, except PublishSelfContained.

That leaves what to do with SelfContained. I think we should just undocument and call it good. I'm not sure that breaking it is useful. Note that this is separate from the CLI args, which should stay as-is.

[agocke]

I think we should just undocument and call it good

We'll need a new command line argument then, and we'll need to change error messages

[agocke]

That is, you'll get an error message from AOT and trimming saying you need to enable self-contained. We need to instead say, PublishSelfContained or -p:PublishSelfContained

[richlander]

I think we're getting at CLI and project file UX being different.

The following seems fine to me:

dotnet publish -p:PublishTrimmed --sc

This seems like way too much:

dotnet publish -p:PublishTrimmed -p:PublishSelfContained

If we want to leave SelfContained alone, that's fine. I was trying to re-raise a suggestion you had raised earlier. Do you have another suggestion on how to approach that?

[agocke]

My preference is that we would change SelfContained to mean PublishSelfContained. If you want a self contained build, you'll need to execute a publish. Basically the same behavior, just behind a different verb.

[richlander]

I see. This change would mean that we are for sure saying that the way to produce final prod apps with .NET is via publish. I think we're mostly there anyway. If no one else wants to raise a concern, I'm good with that. In fact, it might simplify the whole model.

@baronfel @dsplaisted

[marcpopMSFT]

It sounds like we're settling on a plan that is FDD by default always. That means introducing errors when aot, trimming, and singlefile correct? Would dotnet build --sc also produce an error?

+1 to Andy's suggestion of SelfContained meaning PublishSelfContained

Would we need to coordinate changes to the api -aot template to add the PublishSelfContained property? Are there any other templates that would need to be changed (do we have any way of searching the templates for any of the three Publish properties that are affected)?

[richlander]

Mostly "yes" and great questions.

That means introducing errors when aot, trimming, and singlefile correct?

• For .NET 6 apps, no.
• For .NET 7 apps, yes depending on how they are configured.
• For .NET 8 apps, no, since we'll change the templates.

Would dotnet build --sc also produce an error?

With this proposal, yes. We'd move that to dotnet publish only.

Would we need to coordinate changes to the api -aot template

Yes.

This actually aligns quite closely with the -r change now that I think about it. It is just one step further.

[DamianEdwards]

I've read all the reasoning here, and while I don't fault the logic, I find the fact that PublishAot will now simply error without PublishSelfContained also being specified a strange UX for us to default to, especially when we seem to believe it's extremely unlikely we'd ever support AOT with FDD apps.

[richlander]

You are not wrong. That's why there were both "Sven" and "Rich" options.

However, following Jan's logic, this should not be an actual problem. Fixup your project file and it will work.

While my proposal is more pragmatic, I think Sven's proposal is perhaps stronger. We get to document that we offer exactly one model and the project files always look the same way. For example, SCD+Trimming look the same as SCD+AOT. That argument falls down a bit since some properties might infer trimming, but I'm choosing to ignore that.

[dsplaisted]

My preference is that we would change SelfContained to mean PublishSelfContained. If you want a self contained build, you'll need to execute a publish. Basically the same behavior, just behind a different verb.

I see. This change would mean that we are for sure saying that the way to produce final prod apps with .NET is via publish. I think we're mostly there anyway. If no one else wants to raise a concern, I'm good with that. In fact, it might simplify the whole model.

This seems like a major change breaking change. Maybe publish should be the way to get the final prod app, but I bet there are lots of people that don't do it that way today.

One thing that springs to mind is Exe to Exe project references. It's somewhat of a niche scenario, but today you can have a self-contained app reference another self-contained app, and you can get a working copy of both in the output folder. This wouldn't work with publish, since when you publish a project, you don't publish its project references.

[agocke]

This wouldn't work with publish, since when you publish a project, you don't publish its project references.

It works fine though, you just need to pass <ProjectReference Targets="...;Publish" /> instead of a plain ProjectReference. I'd much rather people do that since right now there's a large cliff where if you want just a self-contained build, that can work with no extra configuration, but if you want a single-file build you have to switch to Publish and all the associated configuration.

I'd rather we have one, well-documented way of doing this. And since publish is our gesture going forward for app deployment, we should group this all under publish.

[dsplaisted]

It works fine though, you just need to pass <ProjectReference Targets="...;Publish" /> instead of a plain ProjectReference.

Are you sure that works? There's a protocol for getting the files from a referenced project, and I don't think running the Publish target on the reference would mean that it would return the published files instead of the build output to the referencing project.

[richlander]

I don't have a strong viewpoint on this topic. Perhaps the two of you can hash out a proposal?

[agocke]

I don't have a ProjectReference sample handy so I can't test it, but I don't think it matters because I know this works:

<MSBuild Projects="$(RepoRoot)src/coreclr/tools/aot/crossgen2/crossgen2.csproj"
             Targets="Restore"
             Properties="MSBuildRestoreSessionId=$([System.Guid]::NewGuid())
              ;_IsPublishing=true" />

    <MSBuild Projects="$(RepoRoot)src/coreclr/tools/aot/crossgen2/crossgen2.csproj"
             Targets="Publish;PublishItemsOutputGroup"
             Properties="_IsPublishing=true">
      <Output TaskParameter="TargetOutputs"
              ItemName="_RawCrossgenPublishFiles" />
    </MSBuild>

And that's still a better alternative to using SelfContained ProjectReference. If we want to make sure Publish can work with ProjectReference I'd be fine with fixing it, but I'd also be fine with telling people to just use the MSBuild task. Again, one thing that works in all circumstances is better than multiple things that only work some of the time.

[OptiStrat]

The more I look at it, the more I think the same think about PublishTrimmed.

I've seen several asks for FDD Trimming over time (TerraFx - if I remember correctly is one example, WinUI is another).

@richlander @agocke @vitek-karas as part of this thread discusses the usefulness of FDD Trimming and the likelihood to see this feature implemented: I have a scenario where I would have to xcopy-deploy a relatively large set of relatively simple unpackaged WinUI apps. The C#/WinRT projections alone add ~20 MB to every single app. With CsWinRT 2.0, these projections are trimmable, and it would be great if FDD could benefit from this.

@agocke are the challenges described in linker/issues/2609 applicable to my scenario as well, or do you think it would be easier to support?

[vitek-karas]

The issue dotnet/linker#2609 was actually motivated by WinUI projections assembly (it just doesn't say so specifically). But it's ultimately somewhat different problem.

The expectation is that WinUI assemblies are simple in that they have only references to framework and not user code assemblies - so they act as "leafs" in the analysis in that sense (assuming FDD trimming like picture). So supporting trimming for these would probably be a bit easier, but I would expect it to hit most of the problems described in dotnet/linker#2139 regardless.

The problems described in dotnet/linker#2609 are specifically to support a hybrid world where only some assemblies are trimmed AND not everything is analyzed. The trimmer already supports the case where not everything is trimmed, but it does assume that it will analyze everything. dotnet/linker#2609 tries to explore if it's possible to not analyze most of the app if only a small set of assemblies are to be trimmed. That so far proves to be rather difficult.

[sbomer]

From dotnet/runtime#95496 (comment):

As a user, I should just need to set a single property: PublishAot, and all these other things (SelfContained, RID, PublishTrimmed, PublishSingleFile, etc) are all defaulted to a value that works. I shouldn't have to set 2 properties. We had this conversation last year and came to that agreement. From @nagilson's emailed meeting notes of that meeting:

Here is the conclusion from the meeting.
For PublishReadyToRun, the default will not be SelfContained. This is a breaking change and will be conditioned on the new .NET 8 TFM.
For PublishSingleFile, PublishTrimmed, and PublishAot: PublishSelfContained will be the default. There will be no warning saying you should set SelfContained explicitly.

Now that PublishTrimmed implies PublishSelfContained, can we close this issue @eerhardt @nagilson? I opened a separate issue to track the proposal around fixing the SelfContained behavior (#37780).

[eerhardt]

Retested with SDK v8.0.200-preview.23620.12 and the repro steps in the original post now work correctly. Closing.