Is it just me? Intermittent Errors with Automation on Microsoft Hosted Agents

[ppozeti]

Description

We are experiencing intermittent errors when attempting to automate builds in this repository. Despite trying various approaches, the process is not stable, causing repeated failures.

Steps to Reproduce

1. Attempt to generate an image using Microsoft Hosted Agents.
2. Use older tags which end up failing due to dependency checksum mismatches.
3. Attempt to use the latest main branch, which results in the process hanging at different stages.

Expected Behavior

The automation should consistently create images without errors or hanging, regardless of using older tags or the latest main branch.

Actual Behavior

• Using older tags results in dependencies failing with wrong checksums.
• Using the latest main branch leads to random hangs during the process.
• Out of 10 attempts, only once was an image successfully generated.

Examples

• Using tag win22/20250224.5, encountered the following error related to signature thumbprint mismatch:

    azure-arm.image: Package downloaded in 0.39 seconds
    azure-arm.image: Signature thumbprint do not match expected.
    azure-arm.image: At C:\Program Files\WindowsPowerShell\Modules\ImageHelpers\InstallHelpers.ps1:1018 char:9
    azure-arm.image: +         throw "Signature thumbprint do not match expected."
    azure-arm.image: +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    azure-arm.image:     + CategoryInfo          : OperationStopped: (Signature thumb...match expected.:String) [], RuntimeException
    azure-arm.image:     + FullyQualifiedErrorId : Signature thumbprint do not match expected.

• Using the same tag win22/20250224.5, the process got stuck on:

    azure-arm.image: Installing DockerMsftProvider module

• Using tag win22/20250209.1, encountered another thumbprint mismatch error:

    azure-arm.image: Expand archive 'D:\temp\edgedriver_win64.zip' to 'C:\SeleniumWebDrivers\EdgeDriver' directory
==> azure-arm.image: Signature thumbprint do not match expected.
==> azure-arm.image: At C:\Program Files\WindowsPowerShell\Modules\ImageHelpers\InstallHelpers.ps1:1018 char:9
==> azure-arm.image: +         throw "Signature thumbprint do not match expected."
==> azure-arm.image: +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
==> azure-arm.image:     + CategoryInfo          : OperationStopped: (Signature thumb...match expected.:String) [], RuntimeException
==> azure-arm.image:     + FullyQualifiedErrorId : Signature thumbprint do not match expected.

Environment

• Automation tooling: Microsoft Hosted Agents
• Pipeline used:

trigger: none
  
parameters:  
- name: vmWindowsVersion  
  displayName: 'VM Image Windows Version'  
  type: string  
  default: 'Windows2022'  
  values:  
    - Windows2019  
    - Windows2022  
    - Windows2025  
  
pool:  
  vmImage: 'windows-2022'  

# Dont change this as it will be used as $(Build.BuildNumber). Both Date and Rev are only available here.
name: ${{ parameters.vmWindowsVersion }}$(Date:yyyyMMdd)$(Rev:r)
  
variables:  
- group: build-image-devops-agents
- name: vmWindowsVersion
  value: ${{ parameters.vmWindowsVersion }}
- name: gitBranchOrTagName
  value: main

jobs:
- job: packer
  displayName: 'Generate VM Image'
  timeoutInMinutes: 0

  steps:
  - checkout: none

  - powershell: |
      Write-Host "Cloning $(gitBranchOrTagName)"
      git clone --depth 1 --branch "$(gitBranchOrTagName)" https://github.com/actions/runner-images.git
      
      Set-Location runner-images

      Import-Module .\helpers\GenerateResourcesAndImage.ps1

      GenerateResourcesAndImage -SubscriptionId "$(SubscriptionId)" -ResourceGroupName "$(ImageResourceGroupName)" -ImageType "$(vmWindowsVersion)" -AzureLocation "$(Location)" -ManagedImageName "$(Build.BuildNumber)" -AzureClientId "$(ClientId)" -AzureClientSecret "$(ClientSecret)" -AzureTenantId "$(TenantId)" -ReuseResourceGroup -RestrictToAgentIpAddress -OnError "cleanup" 
    env:
      ClientSecret: $(ClientSecret)
      PACKER_LOG: 1

Additional Context

Please advise if there are any specific configurations or steps we need to follow to ensure stable automation within this repository. Any help or guidance would be greatly appreciated.

Thank you!