Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow runtime packs to be resolved from the packs folder #14044

Closed
dsplaisted opened this issue Oct 8, 2020 · 7 comments · Fixed by #19596
Closed

Allow runtime packs to be resolved from the packs folder #14044

dsplaisted opened this issue Oct 8, 2020 · 7 comments · Fixed by #19596
Assignees
Milestone

Comments

@dsplaisted
Copy link
Member

For targeting packs, the SDK first checks if they are in the dotnet packs folder. If they are it uses them from there, otherwise it generates a PackageDownload item so that they will be downloaded to the packages folder as part of NuGet restore.

Currently, we don't check for runtime packs in the packs folder. This is because we haven't shipped any with the .NET SDK, as they are only used for self-contained apps and there are many different runtime packs for different RuntimeIdentifiers which represent different OS/architecture combinations.

For .NET 6, iOS and Android apps will need runtime packs, and the set of target RuntimeIdentifiers will be more limited. So they would like to deliver runtime packs as part of their optional SDK workloads.

This means we should update the SDK runtime pack resolution logic to first look in the dotnet packs folder and resolve from there if possible, before generating a PackageDownload item to download the runtime pack if necessary.

@dsplaisted dsplaisted added this to the 6.0.1xx milestone Oct 8, 2020
@dagood
Copy link
Member

dagood commented Oct 8, 2020

Sweet! Will this be implemented generally enough to let us start shipping source-built runtime packs? dotnet/source-build#1215

/cc @omajid @tmds

jonathanpeppers added a commit to jonathanpeppers/maui-workload that referenced this issue Feb 12, 2021
This reverts commit 6bd1b0e.

This idea doesn't fully work yet, see:

dotnet/sdk#14044

It partially works for Android, you can do:

    > .\bin\dotnet\dotnet build .\samples\HelloMaui\HelloMaui.Android\HelloMaui.Android.csproj

But it fails if you set `$(RuntimeIdentifier)`:

    > .\bin\dotnet\dotnet build .\samples\HelloMaui\HelloMaui.Android\HelloMaui.Android.csproj -r android.21-arm
    ...
    error NU1101: Unable to find package Microsoft.Android.Runtime.android.21-arm. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages, nuget.org

I think the first one only works because we set
`$(RuntimeIdentifiers)` *plural* by default--targeting two RIDs. There
is some special `restore` logic running when you define
`$(RuntimeIdentifier)` *singular*.
@marek-safar
Copy link
Contributor

/cc @pranavkm

jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 23, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 24, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 25, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 28, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added an `android-aot` workload, in case you want to install this
  support by itself. This is useful, also as a way to include a
  user-friendly description.

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 29, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added an `android-aot` workload, in case you want to install this
  support by itself. This is useful, also as a way to include a
  user-friendly description.

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 30, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android

We can also create a new workload that extends `android`, and includes
AOT support:

    dotnet workload install android-aot

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 30, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android

We can also create a new workload that extends `android`, and includes
AOT support:

    dotnet workload install android-aot

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 30, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android

We can also create a new workload that extends `android`, and includes
AOT support:

    dotnet workload install android-aot

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonpryor pushed a commit to dotnet/android that referenced this issue Jul 6, 2021
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

	dotnet workload install microsoft-android-sdk-full

Shorten this to be:

	dotnet workload install android

We can also create a new workload that extends `android`, and
includes AOT support:

	dotnet workload install android-aot

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
MSBuild target to run `dotnet workload install` commands.  This should
give us more confidence that our workload can be installed -- and that
our MSBuild tests will consume the installed workload.

Other changes:

  * Added our `Microsoft.Android.Runtime.*` packs to
    `WorkloadManifest.json`.  This was mostly omitted by mistake;
    however, runtime packs aren't resolved either due to:
    dotnet/sdk#14044
  
  * Call `setup-test-environment.yaml` in `dotnet_create_msi` job
  
  * Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
  
  * Add `setup-test-environment.yaml` to another build phase
@steveisok
Copy link
Member

@dsplaisted Do you still anticipate being able to roll this out for 6?

@jonathanpeppers
Copy link
Member

I don't think iOS/Android/Maui workloads would be able to work offline without this?

@steveisok
Copy link
Member

I don't think iOS/Android/Maui workloads would be able to work offline without this?

From what I've seen, they would not.

@dsplaisted
Copy link
Member Author

Yes, we're still planning to do this for .NET 6.

@wli3
Copy link

wli3 commented Jul 8, 2021

note from workload meeting: This is a rather easy change. At the same time, not having the feature in not end of the world (use library pack with waste disk space). We should pick up this task after –architecture os work,

pjcollins added a commit to pjcollins/xamarin-macios that referenced this issue Jul 21, 2021
Runtime pack entries should be included in all workload manifest files,
even though they will not yet be resolved from the packs folder due to
dotnet/sdk#14044.  These changes will also
allow us to start producing .msi files for the runtime packs for an
eventual inclusion in the Visual Studio installer.
rolfbjarne pushed a commit to xamarin/xamarin-macios that referenced this issue Jul 22, 2021
Runtime pack entries should be included in all workload manifest files,
even though they will not yet be resolved from the packs folder due to
dotnet/sdk#14044.  These changes will also
allow us to start producing .msi files for the runtime packs for an
eventual inclusion in the Visual Studio installer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants