Description
Note: This is a sub-task of #803 for debugging
Scope
This PR is about GC-time heap dumping for debugging.
The requirement is different from user-visible heap traversing API, such as JVM TI and Ruby's ObjectSpace. Those APIs are for users to enumerate objects of certain kinds, or children of certain objects. They expect the GC to be bug-free, otherwise they will not work. But this PR is intended for helping us debugging buggy GC algorithms or buggy VM bindings.
The heap dumping mentioned in this PR shall happen during GC. It may include both
- enumerating objects (live of dead) using the valid-object bit (VO bit), and
- enumerating objects (live only) using transitive closure.
They should complement each other, and the discrepancies between them may reveal bugs.
"Broken edges" problem
Currently, if any edge (root or field) contains a dangling pointer, the GC will either crash with SIGSEGV or result in an assertion failure. This is too fast, because when we are debugging, we want to dump as many objects as possible so that we can get a "big picture" of the VM state.
To solve this problem, we can let object tracing "fail slow". When tracing an edge (root or field), we can use the VO bit to check whether it points to a valid object or not. If not, it should record the "broken" edge instead of crashing. It shall write the heap dump into a file before panicking if there are any broken edges.
This feature is costly because it requires a VO-bit check for every edge. So it should be able to be turned off at compile time. We can make it a Cargo feature because it also depends on the vo_bit Cargo feature.