Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WS2025 VHD #1527

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Add WS2025 VHD #1527

wants to merge 1 commit into from

Conversation

TinaMor
Copy link

@TinaMor TinaMor commented Jul 25, 2024

PR Description

Server Image: Windows Server 2025 Preview

Adds Windows Server 2025 as a VHD and SIG target. This will allow us to add Windows Server 2025 to the Azure marketplace.

  1. make build-azure-sig-windows-2025-containerd
  2. make build-azure-vhd-windows-2025-containerd - NOT IMPLEMENTED

Why is build VHD not implemented?

Creating VMs from unmanaged disks will no longer be possible in the near future.

When we build VHDs with the command, make build-azure-vhd-windows-2025-containerd, we get a build error:

The specified platform image is not supported for creating a Virtual Machine with unmanaged disks. Please refer to the disallowed VM disk types at https://docs.microsoft.com/en-us/rest/api/compute/virtualmachineimages/get#vmdisktypes.

Packer is displaying the following warning:

azure-arm.vhd-windows-2025-containerd: Warning: You are using Azure Packer Builder to create VHDs which is being deprecated, consider using Managed Images. Learn more https://www.packer.io/docs/builders/azure/arm#azure-arm-builder-specific-options

According to Microsoft documentation:

As of January 30, 2024, new customers won't be able to create unmanaged disks.
We're retiring Azure unmanaged disks by September 30, 2025 - Azure Virtual Machines | Microsoft Learn

There is no workaround for creating a VHD with the WS2025 preview image, and creating an unmanaged disk with this preview image on the Azure Portal isn't possible either. Given the deprecation warnings, consider phasing out the use of unmanaged disks altogether and start preparing for this transition.

Why WS2025 purchase plan information added?

When creating a VM with the generated image, plan information is required.

Example:

$PLAN_PUBLISHER="microsoftwindowsserver" # Plan publisher
$PLAN_PRODUCT="microsoftserveroperatingsystems-previews" # Product ID/ Product Name/ Plan product/ Image Offer
$PLAN_NAME="windows-server-2025-azure-edition-hotpatch" # Plan ID/ Plan Name/ SKU

az vm create `
    --accept-term `
    --resource-group $ResourceGroupName `
    --name $VMName `
    --image $imagedefinition.Id `
    --license-type $LicenseType `
    --location $Location `
    --admin-username $VMLocalAdminUser `
    --admin-password $VMLocalAdminSecurePassword `
    --computer-name $ComputerName `
    --plan-name $PLAN_NAME `
    --plan-product $PLAN_PRODUCT `
    --plan-publisher $PLAN_PUBLISHER `
    --security-type $SecurityType `
    --size $VMSize

Known issues with Windows Server 2025 Preview Image

  1. Failing test: Key Vault gMSA CCG interface is registered

    gMSA test fails because the CCG Interface value is missing. We skip this test for Windows Server 2025 images only

  2. Failing test: Test Windows Features Installed
    The test fails on the first run but passes on rerun. We add a retry argument when calling the goss validate command

@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 25, 2024
Copy link

linux-foundation-easycla bot commented Jul 25, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: TinaMor / name: Tina Murimi (611eefd)

@k8s-ci-robot k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jul 25, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign drew-viles for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
Copy link
Contributor

Welcome @TinaMor!

It looks like this is your first PR to kubernetes-sigs/image-builder 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/image-builder has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot
Copy link
Contributor

Hi @TinaMor. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 25, 2024
@jsturtevant
Copy link
Contributor

@TinaMor could you take a look at the CLA? Thanks!

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jul 28, 2024
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 29, 2024
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 29, 2024
@jsturtevant
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 29, 2024
Comment on lines 16 to 23
export AZURE_LOCATION="${AZURE_LOCATION:-southcentralus}"

# Creating Azure VMs from a Marketplace Image requires a Purchase Plan
# https://portal.azure.com/#view/Microsoft_Azure_Marketplace/GalleryItemDetailsBladeNopdl/id/microsoftwindowsserver.microsoftserveroperatingsystems-previews/selectionMode~/false/resourceGroupId//resourceGroupLocation//dontDiscardJourney~/false
export PLAN_PUBLISHER="${PLAN_PUBLISHER:-''}"
export PLAN_OFFER="${PLAN_OFFER:-''}"
export PLAN_NAME="${PLAN_NAME:-''}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the need for this change. We create other sigs (ws2019 and ws2022) without these values. Where you getting errors?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you try to create an AzVM with the generated image without the plan information we get:

{
    "code": "VMMarketplaceInvalidInput",
    "message": "Unable to deploy from the Marketplace image or a custom image sourced from Marketplace image. The part number in the purchase information for VM '/subscriptions/xxxx/resourceGroups/cluster-api-images/providers/Microsoft.Compute/virtualMachines/xxxx' is not as expected. Beware that the Plan object's properties are case-sensitive. "
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error should go away when we switch to the 2025 GA images so at that point we can drop PLAN information added to workaround this issue for the preview images

@jsturtevant
Copy link
Contributor

looks like the VHD failed with:

1;32m==> azure-arm.vhd-windows-2025-containerd: ERROR: -> DeploymentFailed : At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.
==> azure-arm.vhd-windows-2025-containerd: ERROR:   -> Conflict
==> azure-arm.vhd-windows-2025-containerd: ERROR:   -> OperationNotAllowed : The specified platform image is not supported for creating a Virtual Machine with unmanaged disks. Please refer to the disallowed VM disk types at https://docs.microsoft.com/en-us/rest/api/compute/virtualmachineimages/get#vmdisktypes. 
==> azure-arm.vhd-windows-2025-containerd:�[0m
�[1;31m==> azure-arm.vhd-windows-2025-containerd: Code="DeploymentFailed" Message="At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details." Details=[{"code":"Conflict","message":"{\r\n  \"error\": {\r\n    \"code\": \"OperationNotAllowed\",\r\n    \"message\": \"The specified platform image is not supported for creating a Virtual Machine with unmanaged disks. Please refer to the disallowed VM disk types at https://docs.microsoft.com/en-us/rest/api/compute/virtualmachineimages/get#vmdisktypes. \"\r\n  }\r\n}"}]�[0m
�[1;32m==> azure-arm.vhd-windows-2025-containerd: 

@TinaMor TinaMor force-pushed the tinamor/dev branch 2 times, most recently from 71d6aea to d42a18a Compare August 27, 2024 10:11
@TinaMor TinaMor marked this pull request as ready for review August 27, 2024 10:25
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 27, 2024
@TinaMor TinaMor marked this pull request as draft August 27, 2024 12:16
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 27, 2024
@TinaMor TinaMor marked this pull request as ready for review August 28, 2024 18:30
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 28, 2024
@TinaMor TinaMor force-pushed the tinamor/dev branch 3 times, most recently from ed0e486 to 08cb993 Compare August 29, 2024 11:15
@jsturtevant
Copy link
Contributor

/retest

@jsturtevant
Copy link
Contributor

that looks like the vm had issues booting (maybe quoata?)
/retest

@TinaMor
Copy link
Author

TinaMor commented Sep 2, 2024

/retest

@jsturtevant
Copy link
Contributor

interesting the powershell commands are consistently timing out

�[0;32m    azure-arm.sig-windows-2025-containerd:             "successful": false,�[0m
�[0;32m    azure-arm.sig-windows-2025-containerd:             "summary-line": "Windows Feature - Containers: exit-status: Error: Command execution timed out (1m0s)",�[0m

@jsturtevant
Copy link
Contributor

there are 5 failures, 4 are:

{{ if eq .Vars.OS "windows"}} # Windows
# Workaround until windows features are added to goss
command:
{{range $name, $vers := index .Vars .Vars.OS "common-windows-features"}}
"Windows Feature - {{ $name }}":
exec: powershell -command "(Get-WindowsFeature {{ $name }} | select *)"
exit-status: 0
stdout: {{range $vers.expected}}
- {{.}}
timeout: 60000
{{end}}
{{end}}
{{end}}

For some reason, Get-WindowsFeature is timing out.

the last one is the gmsa registry, the gmsa registry is a known issue and we see it in #1353

@TinaMor TinaMor marked this pull request as draft September 9, 2024 12:46
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 9, 2024
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 10, 2024
@TinaMor
Copy link
Author

TinaMor commented Sep 10, 2024

/test pull-azure-sigs

@iankingori
Copy link

/retest

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Sep 11, 2024
@TinaMor TinaMor force-pushed the tinamor/dev branch 2 times, most recently from 354b8c9 to 6b90c93 Compare September 11, 2024 10:58
@TinaMor
Copy link
Author

TinaMor commented Sep 11, 2024

/retest

    - Adds commands for Windows Server 2025 SIG images
    - Adds Purchase Plan info for Azure Market Place Images

 #   - Adds Purchase Plan info for Azure Market Place Images
@TinaMor
Copy link
Author

TinaMor commented Sep 11, 2024

/retest

@k8s-ci-robot
Copy link
Contributor

@TinaMor: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-ova-all fe35fb5 link false /test pull-ova-all
pull-azure-sigs 611eefd link false /test pull-azure-sigs

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants