Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Long Running Test: Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh #73040

Closed
runfoapp bot opened this issue Jul 29, 2022 · 7 comments · Fixed by #88855
Closed

Long Running Test: Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh #73040

runfoapp bot opened this issue Jul 29, 2022 · 7 comments · Fixed by #88855
Assignees
Milestone

Comments

@runfoapp
Copy link

runfoapp bot commented Jul 29, 2022

Failures 7/5-8/5 (Runfo for last 30 days):

Day Run Details
7/28 PR 1909725 mono OSX x64 Release @ OSX.1200.Amd64.Open
7/28 Rolling run 1909685 mono OSX x64 Release @ OSX.1200.Amd64.Open

Note: The other failures associated by Runfo are unrelated to this issue.

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-71203-merge-9a2f7270283440c1b0/Interop/1/console.18899052.log?%3Fhelixlogtype%3Dresult

    Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh [FAIL]
      
      cmdLine:/private/tmp/helix/working/9BC3084E/w/B4EC0991/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 7/28/2022 1:44:10 PM, end: 7/28/2022 1:54:10 PM)
      
      Return code:      -100
      Raw output file:      /tmp/helix/working/9BC3084E/w/B4EC0991/uploads/Reports/Interop.MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.output.txt
      Raw output:
      BEGIN EXECUTION
      /tmp/helix/working/9BC3084E/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false PInvokeDetach.dll ''
      
      cmdLine:/private/tmp/helix/working/9BC3084E/w/B4EC0991/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 7/28/2022 1:44:10 PM, end: 7/28/2022 1:54:10 PM)
      Test Harness Exitcode is : -100
      To run the test:
      > set CORE_ROOT=/tmp/helix/working/9BC3084E/p
      > /private/tmp/helix/working/9BC3084E/w/B4EC0991/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh
      Expected: True
      Actual:   False
      Stack Trace:
           at Interop_MonoAPI._MonoMono_PInvokeDetach_PInvokeDetach_._MonoMono_PInvokeDetach_PInvokeDetach_sh()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
      Output:
        
        cmdLine:/private/tmp/helix/working/9BC3084E/w/B4EC0991/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 7/28/2022 1:44:10 PM, end: 7/28/2022 1:54:10 PM)
        
        Return code:      -100
        Raw output file:      /tmp/helix/working/9BC3084E/w/B4EC0991/uploads/Reports/Interop.MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.output.txt
        Raw output:
        BEGIN EXECUTION
        /tmp/helix/working/9BC3084E/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false PInvokeDetach.dll ''
        
        cmdLine:/private/tmp/helix/working/9BC3084E/w/B4EC0991/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 7/28/2022 1:44:10 PM, end: 7/28/2022 1:54:10 PM)
        Test Harness Exitcode is : -100
        To run the test:
        > set CORE_ROOT=/tmp/helix/working/9BC3084E/p
        > /private/tmp/helix/working/9BC3084E/w/B4EC0991/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh
  Finished:    Interop.MonoAPI.XUnitWrapper

Runfo Tracking Issue: interop work item

Build Definition Kind Run Name Console Core Dump Test Results Run Client
338124 runtime PR 88695 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log runclient.py
338039 runtime PR 88695 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log runclient.py
336451 runtime PR 88695 coreclr windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
336451 runtime PR 88695 coreclr windows arm64 Checked no_tiered_compilation @ Windows.11.Arm64.Open console.log runclient.py
336451 runtime PR 88695 coreclr windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
336451 runtime PR 88695 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log runclient.py
335829 runtime PR 88695 coreclr windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
335829 runtime PR 88695 coreclr windows arm64 Checked no_tiered_compilation @ Windows.11.Arm64.Open console.log runclient.py
335829 runtime PR 88695 coreclr windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
335829 runtime PR 88695 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log runclient.py
334951 runtime Rolling mono OSX x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
334693 runtime PR 88641 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
334367 runtime PR 88623 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
332801 runtime PR 88415 mono windows x64 Release @ Windows.10.Amd64.Open console.log runclient.py
332801 runtime PR 88415 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
332317 runtime PR 88415 mono windows x64 Release @ Windows.10.Amd64.Open console.log runclient.py
330495 runtime PR 87141 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
330093 runtime PR 88415 mono windows x64 Release @ Windows.10.Amd64.Open console.log runclient.py
329697 runtime PR 88415 mono windows x64 Release @ Windows.10.Amd64.Open console.log runclient.py
329565 runtime PR 88415 mono windows x64 Release @ Windows.10.Amd64.Open console.log runclient.py
329565 runtime PR 88415 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
329356 runtime PR 88345 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
328288 runtime PR 88075 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
327911 runtime PR 87865 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
322889 runtime PR 87903 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
321595 runtime PR 88034 coreclr osx x64 Checked no_tiered_compilation @ OSX.1200.Amd64.Open console.log runclient.py
321595 runtime PR 88034 coreclr osx arm64 Checked no_tiered_compilation @ OSX.1200.ARM64.Open console.log runclient.py
321153 runtime PR 87785 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log core dump runclient.py
320488 runtime PR 88034 coreclr osx arm64 Checked no_tiered_compilation @ OSX.1200.ARM64.Open console.log runclient.py
320488 runtime PR 88034 coreclr osx x64 Checked no_tiered_compilation @ OSX.1200.Amd64.Open console.log runclient.py
316639 runtime PR 87912 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
315985 runtime PR 87878 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
313997 runtime PR 87822 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
313454 runtime PR 87696 mono OSX x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
313238 runtime PR 87801 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
313130 runtime PR 87798 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
312991 runtime PR 87148 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log core dump runclient.py
312987 runtime PR 87148 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log core dump runclient.py
311696 runtime PR 87148 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log core dump runclient.py
311332 runtime PR 87728 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
309227 runtime PR 87148 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log core dump runclient.py
309057 runtime Rolling mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
307725 runtime PR 87148 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log core dump runclient.py
307328 runtime PR 87148 coreclr windows x64 Checked @ Windows.10.Amd64.Open console.log core dump runclient.py
306947 runtime PR 87555 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py
305960 runtime PR 87522 mono osx x64 Release @ OSX.1200.Amd64.Open console.log runclient.py

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
2 9 36
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jul 29, 2022
@noahfalk noahfalk added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Jul 29, 2022
@steveisok
Copy link
Member

/cc @SamMonoRT

@SamMonoRT SamMonoRT added this to the 7.0.0 milestone Aug 3, 2022
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Aug 3, 2022
@karelz karelz changed the title interop work item Long Running Test: Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Aug 5, 2022
@karelz
Copy link
Member

karelz commented Aug 5, 2022

6 unique hits per Runfo table above as of 8/5, incl. 1 Rolling run - 1909685 on 7/28 AM (2nd run)

@BrzVlad
Copy link
Member

BrzVlad commented Aug 5, 2022

I can't reproduce this or investigate it. Also this hang seems to have happened only once. I think we should close this for now.

@karelz karelz removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Aug 5, 2022
@karelz
Copy link
Member

karelz commented Aug 5, 2022

It happened twice - see updated top post (from manual search of last 30 days of failures).
Most of the Runfo associated failures are different problems - I wish there was a way to disassociate the Runfo failure with the issue.

Removing 'blocking-clean-ci' label as it does not happen that often yet.

@BrzVlad
Copy link
Member

BrzVlad commented Aug 5, 2022

If the crash dump would contain the runtime stacktrace, fixing this might be pretty easy. Not sure if there is a problem in the infrastructure that prevents this. Moving to 8.0 for now since it doesn't look like I will be able to investigate this anytime soon.

@BrzVlad BrzVlad modified the milestones: 7.0.0, 8.0.0 Aug 5, 2022
@carlossanlop
Copy link
Member

Seen in an unrelated release/7.0-rc2 backport PR: #75675

Queues:

Callstack
Attempting to collect crash dump: /cores/crashdump_34082.dmp
Invoking: sudo /tmp/helix/working/C04709ED/p/createdump --name "/cores/crashdump_34082.dmp" 34082 --withheap
createdump stdout:
[createdump] Gathering state for process 34082 
[createdump] Target process is alive

createdump stderr:
[createdump] InitializeDAC: CLRDataCreateInstance(ICLRDataEnumMemoryRegions) FAILED The debuggee memory space does not have the expected debugging export table (80131c4f)
[createdump] Failure took 2180ms

Failed to collect crash dump
    Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh [FAIL]
      
      cmdLine:/private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 9/19/2022 12:05:10 PM, end: 9/19/2022 12:15:10 PM)
      
      Return code:      -100
      Raw output file:      /tmp/helix/working/C04709ED/w/B2D3098D/uploads/Reports/Interop.MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.output.txt
      Raw output:
      BEGIN EXECUTION
      /tmp/helix/working/C04709ED/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false PInvokeDetach.dll ''
      
      cmdLine:/private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 9/19/2022 12:05:10 PM, end: 9/19/2022 12:15:10 PM)
      Test Harness Exitcode is : -100
      To run the test:
      > set CORE_ROOT=/tmp/helix/working/C04709ED/p
      > /private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh
      Expected: True
      Actual:   False
      Stack Trace:
           at Interop_MonoAPI._MonoMono_PInvokeDetach_PInvokeDetach_._MonoMono_PInvokeDetach_PInvokeDetach_sh()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
      Output:
        
        cmdLine:/private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 9/19/2022 12:05:10 PM, end: 9/19/2022 12:15:10 PM)
        
        Return code:      -100
        Raw output file:      /tmp/helix/working/C04709ED/w/B2D3098D/uploads/Reports/Interop.MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.output.txt
        Raw output:
        BEGIN EXECUTION
        /tmp/helix/working/C04709ED/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false PInvokeDetach.dll ''
        
        cmdLine:/private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 9/19/2022 12:05:10 PM, end: 9/19/2022 12:15:10 PM)
        Test Harness Exitcode is : -100
        To run the test:
        > set CORE_ROOT=/tmp/helix/working/C04709ED/p
        > /private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh
Callstack
Attempting to collect crash dump: /cores/crashdump_34082.dmp
Invoking: sudo /tmp/helix/working/C04709ED/p/createdump --name "/cores/crashdump_34082.dmp" 34082 --withheap
createdump stdout:
[createdump] Gathering state for process 34082 
[createdump] Target process is alive

createdump stderr:
[createdump] InitializeDAC: CLRDataCreateInstance(ICLRDataEnumMemoryRegions) FAILED The debuggee memory space does not have the expected debugging export table (80131c4f)
[createdump] Failure took 2180ms

Failed to collect crash dump
    Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh [FAIL]
      
      cmdLine:/private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 9/19/2022 12:05:10 PM, end: 9/19/2022 12:15:10 PM)
      
      Return code:      -100
      Raw output file:      /tmp/helix/working/C04709ED/w/B2D3098D/uploads/Reports/Interop.MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.output.txt
      Raw output:
      BEGIN EXECUTION
      /tmp/helix/working/C04709ED/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false PInvokeDetach.dll ''
      
      cmdLine:/private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 9/19/2022 12:05:10 PM, end: 9/19/2022 12:15:10 PM)
      Test Harness Exitcode is : -100
      To run the test:
      > set CORE_ROOT=/tmp/helix/working/C04709ED/p
      > /private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh
      Expected: True
      Actual:   False
      Stack Trace:
           at Interop_MonoAPI._MonoMono_PInvokeDetach_PInvokeDetach_._MonoMono_PInvokeDetach_PInvokeDetach_sh()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
      Output:
        
        cmdLine:/private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 9/19/2022 12:05:10 PM, end: 9/19/2022 12:15:10 PM)
        
        Return code:      -100
        Raw output file:      /tmp/helix/working/C04709ED/w/B2D3098D/uploads/Reports/Interop.MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.output.txt
        Raw output:
        BEGIN EXECUTION
        /tmp/helix/working/C04709ED/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false PInvokeDetach.dll ''
        
        cmdLine:/private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 9/19/2022 12:05:10 PM, end: 9/19/2022 12:15:10 PM)
        Test Harness Exitcode is : -100
        To run the test:
        > set CORE_ROOT=/tmp/helix/working/C04709ED/p
        > /private/tmp/helix/working/C04709ED/w/B2D3098D/e/Interop/MonoAPI/MonoMono/PInvokeDetach/PInvokeDetach.sh

@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 13, 2023
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 13, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Aug 14, 2023
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants