MDEV-36316/MDEV-36327/MDEV-36328 Debug msan fixes 10.6 #3899
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In CMAKE_BUILD_TYPE=Debug the MSAN of clang-20.1 results in MemorySanitizer: use-of-uninitialized-value on mach_read_from_2 called by rec_set_bit_field_2 (and likewise for the _1 equivalent). The non-debug builds are assumed to optimize this down such that this becomes just a setting of values.
grooverdan
added
the
MariaDB Foundation
grooverdan
changed the title
Without this increase the mtr test case pre/post conditions will fail as the stack usage has increased under MSAN with clang-20.1. A partial success with 432K was achieved, however the 448K was needed for test cases that changed default collation. The resulting behaviour observed on smaller stack size was SEGV when a function allocated memory from the stack, and the called another function (potentially coincidenly memset - assuming common in early functions post allocation).
grooverdan
force-pushed
the
debug-msan-fixes-10.6
branch
from
12611e9 to
b445f66
Compare
The function dict_process_sys_columns_rec left nth_v_col uninitialized unless it was a virtual column. This was ok as the function i_s_sys_columns_fill_table also didn't read this value unless it was a virtual column. As MSAN in clang-20 didn't follow this though, the pass by value was changed to a pass by ptr so that MSAN could detect this correctly.
…n_range ror_scan_selectivity passed an uninitialized page structure so we shouldn't be using its values. btr_estimate_n_rows_in_range doesn't use the page numbers in the tuples so these can be omitted. While ror_scan_selectivity never uses the result, however the mrr calling of records_in_range does use the result.
grooverdan
force-pushed
the
debug-msan-fixes-10.6
branch
from
b445f66 to
5e9b106
Compare
dr-m
reviewed
Mar 21, 2025
Comment on lines
+159
to
164
| #ifndef DBUG_OFF | ||
| MEM_MAKE_DEFINED(rec - offs, 1); | ||
| #endif | ||
| mach_write_to_1(rec - offs, | ||
| (mach_read_from_1(rec - offs) & ~mask) | ||
| | (val << shift)); |
There was a problem hiding this comment.
This looks incorrect to me. Why would we claim that all bits at rec[-offs] are initialized when we are only overwriting some of the bits here? What would fail if this change and the similar change to rec_set_bit_field_2() were omitted?
Comment on lines
-5456
to
5457
| ulint nth_v_col, /*!< in: virtual column, its | ||
| ulint* nth_v_col, /*!< in: virtual column, its | ||
| sequence number (nth virtual col) */ |
There was a problem hiding this comment.
I do not understand why we would need any of the changes to this file and which problem these changes would solve. We’re no longer passing a read-only parameter by value but via a pointer that is effectively read-only. Can you test again without including any of these changes?
Comment on lines
-14479
to
+14480
| btr_pos_t tuple1(range_start, mode1, pages->first_page); | ||
| btr_pos_t tuple2(range_end, mode2, pages->last_page); | ||
| btr_pos_t tuple1(range_start, mode1, 0); | ||
| btr_pos_t tuple2(range_end, mode2, 0); |
There was a problem hiding this comment.
This would seem to be the actual fix. ~0ULL might be a safer value, but I think that 0 should be OK as well, because the smallest possible index page number is 3.
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Various fixing to allow a Debug MSAN with Clang-20 to pass tests.
See individual commit messages for details.
Release Notes
How can this PR be tested?
Basing the PR against the correct MariaDB version
mainbranch.PR quality check