Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[wasm] Assertion: should not be reached at D:/a/_work/1/s/src/mono/mono/sgen/sgen-scan-object.h:93 in System.Reflection.Tests #95805

Closed
radical opened this issue Dec 8, 2023 · 8 comments
Assignees
Labels
arch-wasm WebAssembly architecture area-GC-mono Known Build Error Use this to report build issues in the .NET Helix tab
Milestone

Comments

@radical
Copy link
Member

radical commented Dec 8, 2023

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=629971
Build error leg or test failing: WasmTestOnBrowser-System.Linq.Expressions.Tests.WorkItemExecution
Pull request: #95738

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Assertion: should not be reached at .*src/mono/mono/sgen/sgen-scan-object.h:93",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Log

[08:32:20] fail: [0x05bf02f0-long 08:32:20.979] [MONO] * Assertion: should not be reached at /__w/1/s/src/mono/mono/sgen/sgen-scan-object.h:93
                 
                 Error
                     at cu (http://127.0.0.1:46083/_framework/dotnet.runtime.js:3:201010)
                     at wasm_trace_logger (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[199]:0x14ac1)
                     at eglib_log_adapter (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[693]:0x4bdf2)
                     at monoeg_g_logv_nofree (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[615]:0x4a203)
                     at monoeg_assertion_message (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[619]:0x4a324)
                     at mono_assertion_message_unreachable (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[622]:0x4a38d)
                     at drain_gray_stack (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[1042]:0x5b8b8)
                     at finish_gray_stack (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[958]:0x573b6)
                     at major_do_collection (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[932]:0x553f9)
                     at sgen_perform_collection (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[928]:0x54543)
                     at sgen_gc_collect (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[945]:0x5674a)
                     at mono_gc_collect (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[3189]:0xf1e7f)
                     at ves_icall_System_GC_InternalCollect (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[3118]:0xee579)
                     at do_icall (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[338]:0x28e37)
                     at do_icall_wrapper (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[267]:0x2648f)
                     at mono_interp_exec_method (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[258]:0x18d33)
                     at interp_runtime_invoke (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[297]:0x2711d)
                     at mono_jit_runtime_invoke (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[3349]:0xfd935)
                     at do_runtime_invoke (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[2629]:0xd01b2)
                     at mono_runtime_invoke_checked (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[2628]:0xd0142)
                     at ves_icall_InternalInvoke_raw (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[1985]:0xaab63)
                     at do_icall (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[338]:0x28edf)
                     at do_icall_wrapper (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[267]:0x2648f)
                     at mono_interp_exec_method (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[258]:0x18d33)
                     at interp_runtime_invoke (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[297]:0x2711d)
                     at mono_jit_runtime_invoke (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[3349]:0xfd935)
                     at do_runtime_invoke (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[2629]:0xd01b2)
                     at mono_runtime_invoke_checked (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[2628]:0xd0142)
                     at start_wrapper (http://127.0.0.1:46083/_framework/dotnet.native.wasm:wasm-function[2777]:0xd735b)
                     at Object.invokeEntryPoint (http://127.0.0.1:46083/_framework/dotnet.native.js:8:69009)
                     at handleMessage (http://127.0.0.1:46083/_framework/dotnet.native.worker.js:1:2306)

cc @vargaz @lambdageek

Report

Build Definition Test Pull Request
671701 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution
670314 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #100266
670311 dotnet/runtime WasmTestOnFirefox-MT-System.Collections.Concurrent.Tests.WorkItemExecution #100266
670263 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution
669665 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101871
670251 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101681
670248 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution #102039
669355 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution #101977
667200 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101934
667292 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101337
666657 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101717
666620 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101844
666493 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101681
666136 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101843
665654 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution #101843
665447 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution #101871
664531 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101842
664390 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution
664211 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101162
664022 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101828
663140 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution #101716
662281 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution
661662 dotnet/runtime WasmTestOnChrome-MT-System.Collections.Concurrent.Tests.WorkItemExecution #100266
661154 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101715
661111 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution
661068 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101712
660999 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #100266
661010 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101693
661006 dotnet/runtime WasmTestOnFirefox-MT-System.Threading.Tasks.Tests.WorkItemExecution #101693
660346 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101162
660018 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101318
659851 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution
659547 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution #101529
659097 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101672
658322 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101653
658219 dotnet/runtime WasmTestOnChrome-System.Memory.Tests.WorkItemExecution
657933 dotnet/runtime WasmTestOnChrome-MT-System.Collections.Concurrent.Tests.WorkItemExecution
657224 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.Tests.WorkItemExecution #101599
656949 dotnet/runtime WasmTestOnFirefox-MT-System.Runtime.InteropServices.JavaScript.Tests.WorkItemExecution #101532
656887 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution
656594 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #101538
656049 dotnet/runtime WasmTestOnChrome-MT-System.Runtime.Tests.WorkItemExecution #100141
655885 dotnet/runtime WasmTestOnChrome-System.Collections.Concurrent.Tests.WorkItemExecution #101378
655832 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101330
655742 dotnet/runtime WasmTestOnChrome-System.Collections.Concurrent.Tests.WorkItemExecution #101532
655571 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101392
655262 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101460
655218 dotnet/runtime WasmTestOnV8-System.Text.Json.Tests.WorkItemExecution #101318
655024 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101509
653322 dotnet/runtime WasmTestOnChrome-System.Text.Json.Tests.WorkItemExecution #101188
653807 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101442
653607 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101397
653383 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101330
653346 dotnet/runtime WasmTestOnV8-System.Text.Json.Tests.WorkItemExecution #101425
653343 dotnet/runtime WasmTestOnChrome-System.Text.Json.Tests.WorkItemExecution #101424
652841 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101263
652424 dotnet/runtime WasmTestOnChrome-System.Collections.Concurrent.Tests.WorkItemExecution #101397
652287 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #100697
652019 dotnet/runtime WasmTestOnChrome-System.Runtime.InteropServices.JavaScript.Tests.WorkItemExecution #101381
650948 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101345
650709 dotnet/runtime WasmTestOnChrome-System.Collections.Concurrent.Tests.WorkItemExecution #101339
650699 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101337
650691 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101335
650687 dotnet/runtime WasmTestOnChrome-System.Runtime.Tests.WorkItemExecution #101334

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 64

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=569971
Error message validated: [Assertion: should not be reached at .*src/mono/mono/sgen/sgen-scan-object.h:93]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 2/17/2024 12:38:23 PM UTC

@radical radical added arch-wasm WebAssembly architecture blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' area-VM-meta-mono Known Build Error Use this to report build issues in the .NET Helix tab labels Dec 8, 2023
@ghost
Copy link

ghost commented Dec 8, 2023

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=493316
Build error leg or test failing: WasmTestOnBrowser-System.Linq.Expressions.Tests.WorkItemExecution
Pull request: #95738

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Assertion: should not be reached at .*src/mono/mono/sgen/sgen-scan-object.h:93",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

From wasm-console.log:

[2023-12-08T15:53:39.179Z] [PASS] System.Linq.Expressions.Tests.ConvertCheckedTests.ConvertCheckedULongToNullableByteTest(useInterpreter: False)
[2023-12-08T15:53:39.180Z] [PASS] System.Linq.Expressions.Tests.ConvertCheckedTests.ConvertCheckedULongToNullableByteTest(useInterpreter: True)
Error: [MONO] * Assertion: should not be reached at D:/a/_work/1/s/src/mono/mono/sgen/sgen-scan-object.h:93

    at yt (http://127.0.0.1:49226/_framework/dotnet.runtime.js:3:13089)
    at Ql (http://127.0.0.1:49226/_framework/dotnet.runtime.js:3:181258)
    at wasm_trace_logger (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[121]:0xa563)
    at eglib_log_adapter (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[646]:0x3e9aa)
    at monoeg_g_logv_nofree (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[568]:0x3cec6)
    at monoeg_assertion_message (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[572]:0x3cfe7)
    at mono_assertion_message_unreachable (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[575]:0x3d050)
    at simple_nursery_serial_scan_object (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[1040]:0x55aab)
    at simple_nursery_serial_drain_gray_stack (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[1037]:0x54a87)
    at finish_gray_stack (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[885]:0x49206)
Error: [MONO] D:/a/_work/1/s/src/mono/mono/sgen/sgen-gc.c:3970 <disabled>
    at yt (http://127.0.0.1:49226/_framework/dotnet.runtime.js:3:13089)
    at Ql (http://127.0.0.1:49226/_framework/dotnet.runtime.js:3:181258)
    at wasm_trace_logger (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[121]:0xa563)
    at eglib_log_adapter (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[646]:0x3e9aa)
    at monoeg_g_logv_nofree (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[568]:0x3cec6)
    at monoeg_g_log (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[570]:0x3cf89)
    at monoeg_g_log_disabled (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[571]:0x3cfbc)
    at sgen_stop_world (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[859]:0x4679c)
    at sgen_perform_collection_inner (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[858]:0x465c1)
    at sgen_perform_collection (http://127.0.0.1:49226/_framework/dotnet.native.wasm:wasm-function[856]:0x4651e)

cc @vargaz @lambdageek

Author: radical
Assignees: -
Labels:

arch-wasm, blocking-clean-ci, area-VM-meta-mono, Known Build Error

Milestone: -

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Dec 8, 2023
@pavelsavara pavelsavara removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Mar 4, 2024
@lewing lewing closed this as completed Apr 1, 2024
@dotnet-policy-service dotnet-policy-service bot removed the untriaged New issue has not been triaged by the area owner label Apr 1, 2024
@pavelsavara
Copy link
Member

there is new occurrence Log

@pavelsavara pavelsavara reopened this Apr 4, 2024
@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Apr 4, 2024
@pavelsavara pavelsavara added this to the 9.0.0 milestone Apr 4, 2024
@pavelsavara pavelsavara self-assigned this Apr 4, 2024
@dotnet-policy-service dotnet-policy-service bot removed the untriaged New issue has not been triaged by the area owner label Apr 4, 2024
@pavelsavara pavelsavara changed the title [wasm] Assertion: should not be reached at D:/a/_work/1/s/src/mono/mono/sgen/sgen-scan-object.h:93 in System.Linq.Expressions.Tests [wasm] Assertion: should not be reached at D:/a/_work/1/s/src/mono/mono/sgen/sgen-scan-object.h:93 in System.Reflection.Tests Apr 10, 2024
@pavelsavara
Copy link
Member

all the recent ones are in reflection tests in MT.
cc @BrzVlad any ideas ?

@lewing
Copy link
Member

lewing commented Apr 17, 2024

cc @lambdageek

@pavelsavara
Copy link
Member

Note reflections tests are disabled as active issue now for MT, otherwise this would report many more

Copy link
Contributor

Tagging subscribers to this area: @BrzVlad
See info in area-owners.md if you want to be subscribed.

@pavelsavara
Copy link
Member

pavelsavara commented Apr 25, 2024

This one is single-threaded OOM followed by broken GC, Log

[23:08:00] info: [FAIL] System.Text.Json.Tests.Utf8JsonWriterTests.WriteJsonOnlyWritesToStreamOnDemand_FlushAsync
[23:08:00] info: System.OutOfMemoryException : Out of memory
[23:08:00] info:    at System.GC.AllocateArray[Char](Int32 length, Boolean pinned)
[23:08:00] info:    at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
...
[23:08:00] info: [MONO] * Assertion: should not be reached at /__w/1/s/src/mono/mono/sgen/sgen-scan-object.h:93
[23:08:00] info: 
[23:08:00] info: Error
[23:08:00] info:     at Kc (/root/helix/work/workitem/e/_framework/dotnet.runtime.js:3:171784)
[23:08:00] info:     at wasm_trace_logger (wasm://wasm/00b9ea7a:wasm-function[163]:0xb23f)
[23:08:00] info:     at eglib_log_adapter (wasm://wasm/00b9ea7a:wasm-function[648]:0x438e1)
[23:08:00] info:     at monoeg_g_logv_nofree (wasm://wasm/00b9ea7a:wasm-function[571]:0x41840)
[23:08:00] info:     at monoeg_assertion_message (wasm://wasm/00b9ea7a:wasm-function[574]:0x41931)
[23:08:00] info:     at mono_assertion_message_unreachable (wasm://wasm/00b9ea7a:wasm-function[576]:0x4198b)
[23:08:00] info:     at simple_nursery_serial_scan_object (wasm://wasm/00b9ea7a:wasm-function[1049]:0x5bcad)
[23:08:00] info:     at simple_nursery_serial_drain_gray_stack (wasm://wasm/00b9ea7a:wasm-function[1046]:0x5ac89)
[23:08:00] info:     at finish_gray_stack (wasm://wasm/00b9ea7a:wasm-function[893]:0x4ee72)
[23:08:00] info:     at collect_nursery (wasm://wasm/00b9ea7a:wasm-function[868]:0x4ca0f)
[23:08:00] info:     at sgen_perform_collection_inner (wasm://wasm/00b9ea7a:wasm-function[866]:0x4c2bf)
[23:08:00] info:     at sgen_perform_collection (wasm://wasm/00b9ea7a:wasm-function[864]:0x4c12f)
[23:08:00] info:     at sgen_ensure_free_space (wasm://wasm/00b9ea7a:wasm-function[863]:0x4c0d6)
[23:08:00] info:     at sgen_alloc_obj_nolock (wasm://wasm/00b9ea7a:wasm-function[849]:0x4b4f8)
[23:08:00] info:     at mono_gc_alloc_vector (wasm://wasm/00b9ea7a:wasm-function[3047]:0xe92b9)
[23:08:00] info:     at mono_array_new_specific_internal (wasm://wasm/00b9ea7a:wasm-function[2616]:0xcf478)
[23:08:00] info:     at mono_array_new_specific_checked (wasm://wasm/00b9ea7a:wasm-function[2617]:0xcf4af)

Note, it's not the same stack trace, there are 2 different fails, which means that (possibly another) OOM corrupted the memory for the GC.
Maybe malloc returned -1 and the return value was not tested (for which we don't have the stack trace)

Related to this theory #96546 (comment)

@pavelsavara
Copy link
Member

this seems to be good now, closing. Please open another issue if this appears again

@github-actions github-actions bot locked and limited conversation to collaborators Jun 21, 2024
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
arch-wasm WebAssembly architecture area-GC-mono Known Build Error Use this to report build issues in the .NET Helix tab
Projects
None yet
Development

No branches or pull requests

4 participants