The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
960422	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112828
960415	dotnet/runtime	Send tests to Helix (Unix)	Log
960352	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112728
2648794	dotnet-runtime	Send job to Helix (Unix)	Log
2648777	dotnet-runtime	Send to Helix	Log
960304	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112817
960228	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112479
960191	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112818
2648677	dotnet-runtime	Send job to Helix (Windows)	Log
2648688	dotnet-performance	Send job to Helix (Unix)	Log
960173	dotnet/runtime	Send to Helix	Log
960172	dotnet/runtime	Send to Helix	Log
960189	dotnet/runtime	Send to Helix	Log
960114	dotnet/runtime	Send tests to Helix (Unix)	Log
960102	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112479
960072	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112644
960064	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112809
960069	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112171
2648453	dotnet-runtime	Send job to Helix (Windows)	Log
960054	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112632
960042	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#111472
960036	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112728
960020	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112824
960015	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112740
2648414	dotnet-performance	Send job to Helix (Unix)	Log
959975	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112818
959951	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112168
959957	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112670
959930	dotnet/runtime	Send tests to Helix (Windows)	Log
959831	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112815
959918	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112809
2648321	dotnet-runtime	Send job to Helix (Unix)	Log
959903	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112820
959894	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112753
2648268	dotnet-runtime	Send to Helix	Log
959856	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112817
959851	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112632
959595	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112809
2648212	dotnet-runtime	Send job to Helix (Windows)	Log
959786	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112766
2648187	dotnet-performance	Send job to Helix (Unix)	Log
959805	dotnet/runtime	Send to Helix	Log
959813	dotnet/runtime	Send to Helix	Log
959807	dotnet/runtime	Send to Helix	Log
959782	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112730
959753	dotnet/runtime	Send tests to Helix (Unix)	Log
2648086	dotnet-runtime	Send job to Helix (Windows)	Log
959721	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112705
959717	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112810
959687	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112513
959632	dotnet/runtime	Send tests to Helix (Unix)	Log
2647744	dotnet-dotnet	Download Artifacts	Log
959616	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112799
959497	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112806
959590	dotnet/runtime	Send tests to Helix (Unix)	Log
2647948	dotnet-dotnet	Build (centOSStreamContainer)	Log
2647951	dotnet-runtime	Send job to Helix (Unix)	Log
959581	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112808
2647915	dotnet-arcade-validation	🛡 Guardian: CredScan (Binary)	Log
2647896	dotnet-performance	Send job to Helix (Unix)	Log
959440	dotnet/runtime	Send tests to Helix (Unix)	Log
959483	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112805
959460	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112804
959361	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112796
959342	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#112728
959337	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112799
959357	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112801
959312	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112766
959349	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112800
959322	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112736
959308	dotnet/runtime	Send tests to Helix (Unix)	Log
959317	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112736
959274	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112797
959287	dotnet/runtime	Send tests to Helix (Unix)	Log
959218	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#111505
2647712	dotnet-runtime	Send job to Helix (Windows)	Log
959237	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112795
959226	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112793
959166	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112753
958130	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112504
959152	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112791
2647694	dotnet-performance	Send job to Helix (Unix)	Log
959134	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112789
959092	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112004
959137	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112790
959122	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112730
959065	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#111543
959056	dotnet/runtime	Send tests to Helix (Unix)	Log
959100	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112788
2647656	dotnet-sdk	🟣 Build	Log
959082	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112787
959015	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112168
958989	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#112369
959028	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112786
958097	dotnet/runtime	Send to Helix	Log	dotnet/runtime#112139
2647618	dotnet-extensions	Validate Hosted Pool Information (1ES PT)	Log
958954	dotnet/runtime	Send to Helix	Log
958950	dotnet/runtime	Send tests to Helix (Unix)	Log
958951	dotnet/runtime	Send tests to Helix (Unix)	Log
958944	dotnet/runtime	Send tests to Helix (Unix)	Log
Displaying 100 of 1811 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
18	291	1811

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)