Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NativeAOT support in source-build #2885

Closed
crummel opened this issue Jun 2, 2022 · 13 comments
Closed

NativeAOT support in source-build #2885

crummel opened this issue Jun 2, 2022 · 13 comments
Assignees
Labels
area-product-experience Improvements in the end-user's product experience

Comments

@crummel
Copy link
Contributor

crummel commented Jun 2, 2022

From dotnet/runtime#69611, there is more work here than just building llvm in source-build - we have to make some decisions about how different native packages are linked. Currently NativeAOT will not work in source-build because of the assumptions it makes about these linkages.

@crummel crummel added the area-product-experience Improvements in the end-user's product experience label Jun 2, 2022
@MichaelSimons MichaelSimons moved this to 7.0 Backlog in .NET Source Build Jun 2, 2022
@MichaelSimons
Copy link
Member

[Triage] one suggestion is to try an run it today w/the prebuilt to see how much is functional and what is broken.

@tmds
Copy link
Member

tmds commented Aug 26, 2022

Is NativeAOT a feature that gets delivered from the SDK, or does it ship out-of-band from nuget.org?

I've downloaded preview 7, and for NativeAOT I need a to add a reference to Microsoft.DotNet.ILCompiler.

If this ships out-of-band we can skip compiling it on source-build .NET.
Or do we still need to build a part? Or do we need it to build .NET itself?

cc @omajid

@tmds
Copy link
Member

tmds commented Aug 28, 2022

@jkotas can you answer my questions above?

@jkotas
Copy link
Member

jkotas commented Aug 28, 2022

Is NativeAOT a feature that gets delivered from the SDK, or does it ship out-of-band from nuget.org?

If you specify <PublishAot>true</PublishAot> property in your .csproj, SDK will add native AOT package for you and the package gets downloaded from nuget during project restore.

It is same scheme as what's used for publishing self-contained apps (SDK adds Microsoft.NETCore.App.Runtime.* package reference) or PublishReadyToRun (SDK adds Microsoft.NETCore.App.Crossgen2.* package reference).

I've downloaded preview 7, and for NativeAOT I need a to add a reference to Microsoft.DotNet.ILCompiler.

Adding <PublishAot>true</PublishAot> property to your project file is going to be the recommended way to enable publishing as native aot. It should work in preview 7.

Just like with the crossgen2 package, you can manually add the native aot package reference and it will override the one added by the SDK. The explicit package reference should be only used as last resort (e.g. to workaround bugs in the runtime). It tends to produce version mismatches when people upgrade to new runtime, but forget to bump the aot package version.

The manual package overriding is broken in preview 7. It should be fixed in rc builds.

Or do we need it to build .NET itself?

It is not required to build .NET itself.

We use nativeaot to optimize crossgen2 where possible, but this optimization is disabled for source build currently: https://github.com/dotnet/runtime/blob/main/src/coreclr/tools/aot/crossgen2/crossgen2.csproj#L9

@tmds
Copy link
Member

tmds commented Aug 29, 2022

@jkotas thanks. I think this means we have no need to build NativeAOT support for source-build in .NET 7.

It is same scheme as what's used for publishing self-contained apps (SDK adds Microsoft.NETCore.App.Runtime.* package reference) or PublishReadyToRun (SDK adds Microsoft.NETCore.App.Crossgen2.* package reference).

Source-build doesn't need to build the end-user NativeAOT support.

Like self-contained apps in .NET 6, the SDK uses packages published by Microsoft on nuget.org.

I do think these should become features of source-built .NET (#1215). That's not for .NET 7.

It is not required to build .NET itself.

When we build .NET from source, we still need crossgen2 to perform R2R compilation on the runtime assemblies.

And crossgen2 gets built without NativeAOT to do this.

We use nativeaot to optimize crossgen2 where possible

This applies NativeAOT to crossgen2 application itself. We don't ship it.

This optimization doesn't improve the resulting runtime assemblies after R2R compilation (I assume).

@jkotas
Copy link
Member

jkotas commented Aug 29, 2022

I think this means we have no need to build NativeAOT support for source-build in .NET 7.

I agree.

This optimization doesn't improve the resulting runtime assemblies after R2R compilation (I assume).

Correct.

@tmds
Copy link
Member

tmds commented Sep 14, 2022

If you specify true property in your .csproj, SDK will add native AOT package for you and the package gets downloaded from nuget during project restore.

@jkotas, Microsoft's .NET 7 preview 7 includes ./sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.DotNet.ILCompiler.

Does that mean it doesn't get downloaded from nuget, and we should build it from source?

@jkotas
Copy link
Member

jkotas commented Sep 14, 2022

Microsoft's .NET 7 preview 7 includes ./sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.DotNet.ILCompiler

I have discovered that just a few days ago as well. Related problem is that we are bundling more stuff into the SDK than what should be required (dotnet/runtime#75468 is opened on that).

We either need to figure out how to remove the binaries from what we are bundling into SDK and/or we need to start building the package as part of the source build.

@LakshanF Could you please look into this?

@sbomer
Copy link
Member

sbomer commented Sep 14, 2022

From my point of view, it would make sense to source-build the Microsoft.DotNet.ILCompiler package for 7.0. This would put us into a similar situation to crossgen, where the .target files are also bundled with the SDK (the difference is the crossgen targets actually live in SDK instead of flowing in from runtime).

I agree that we should eventually move the targets out of the bundled package to fix versioning issues, but I would hesitate to do so in 7.0 because the SDK doesn't have a way to import targets from the dynamically-downloaded "runtime packs" (the mechanism used for crossgen). We would need to work around this limitation and replace it with an implicit PackageReference, which could require changes to the targets logic. I'm happy to help in case we do want this to get fixed in 7.0.

@tmds
Copy link
Member

tmds commented Sep 15, 2022

Is it correct that the SDK doesn't actually contain the ILCompiler (that is: the thing that does nativeaot), but it contains parts of the ILCompiler package so the SDK knows how to get the actual ILCompiler from nuget.org?

@sbomer
Copy link
Member

sbomer commented Sep 15, 2022

Yes, roughly. The Microsoft.DotNet.ILCompiler package doesn't include the ILCompiler executable. It has logic for locating the version of the actual compiler bits downloaded by the SDK from nuget. It also contains some MSBuild logic: tasks like ComputeManagedAssembliesToCompileToNative, and .targets that compute the command-line arguments to the compiler.

@crummel
Copy link
Contributor Author

crummel commented Sep 26, 2022

I've summarized what I believe to be the current state of play and the agreed-upon solution for NativeAOT in 7.0 in dotnet/runtime#76206. Please check this out and let me know if you have any concerns.

@MichaelSimons
Copy link
Member

NativeAOT support removed from net 7.0 source-build with dotnet/installer#14697 - closing

Repository owner moved this from WIP to Done in .NET Source Build Oct 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-product-experience Improvements in the end-user's product experience
Projects
Archived in project
Development

No branches or pull requests

5 participants