Skip to content

Add AddressSanitizer to DEBUG builds #1279

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

JackPGreen
Copy link
Contributor

@JackPGreen JackPGreen commented May 15, 2025

#1276 addressed a Segementation fault, but from the error alone, the root cause was not obvious:

31168 Segmentation fault: 11  build/cpp_client

By adding asan to the DEBUG builds, the CI job would've failed with a much more helpful error message:

=================================================================
==37978==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000040 (pc 0x7eff21446d48 bp 0x7efeff67fce0 sp 0x7efeff67fcd0 T22)
==37978==The signal is caused by a READ memory access.
==37978==Hint: address points to the zero page.

    #0 0x7eff21446d48 in hazelcast::logger::enabled(hazelcast::logger::level) /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/hazelcast/src/hazelcast/logger.cpp:71
    #1 0x7eff20a05b1d in hazelcast::client::protocol::codec::client_addclusterviewlistener_handler::handle(hazelcast::client::protocol::ClientMessage&) /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/hazelcast/generated-sources/src/hazelcast/client/protocol/codec/codecs.cpp:145

{...}

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/hazelcast/src/hazelcast/logger.cpp:71 in hazelcast::logger::enabled(hazelcast::logger::level)

Changes:

  • migrate the implementation of the DEBUG builds into the build- scripts
    • e.g. WARN_AS_ERROR can be assumed when DEBUG is enabled, rather than being duplicated in each job configuration
    • remove the architecture to pass these redundant params down
  • pass BUILD_TYPE into the build- scripts as an environment variable, as used for other parameters
  • when the build- scripts are run in DEBUG, also add the asan arguments
    • this is the actual thing that we wanted to change

#1276 addressed a `Segementation fault`, but from the error alone, the root cause was not obvious:
```
31168 Segmentation fault: 11  build/cpp_client
```

By adding [`asan`](https://learn.microsoft.com/en-us/cpp/sanitizers/asan) to the `DEBUG` builds, the CI job would've failed with a much more helpful error message:
```
=================================================================
==37978==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000040 (pc 0x7eff21446d48 bp 0x7efeff67fce0 sp 0x7efeff67fcd0 T22)
==37978==The signal is caused by a READ memory access.
==37978==Hint: address points to the zero page.

    #0 0x7eff21446d48 in hazelcast::logger::enabled(hazelcast::logger::level) /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/hazelcast/src/hazelcast/logger.cpp:71
    #1 0x7eff20a05b1d in hazelcast::client::protocol::codec::client_addclusterviewlistener_handler::handle(hazelcast::client::protocol::ClientMessage&) /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/hazelcast/generated-sources/src/hazelcast/client/protocol/codec/codecs.cpp:145

{...}

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/hazelcast/src/hazelcast/logger.cpp:71 in hazelcast::logger::enabled(hazelcast::logger::level)
```

An _alternative_ implementation would've been to hardcode the flags in the `build-unix`/`build-windows` instead.
@JackPGreen JackPGreen self-assigned this May 15, 2025
JackPGreen and others added 4 commits May 16, 2025 22:39
Co-authored-by: ihsan demir <ihsandemir@gmail.com>
Co-authored-by: ihsan demir <ihsandemir@gmail.com>
@JackPGreen
Copy link
Contributor Author

@ihsandemir thanks for your feedback. I was actually waiting for a green PR builder before un-drafting this PR but it seems GitHub gave you a heads-up.

The good news is that asan has already spotted some issues:

==37969==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 48 byte(s) in 2 object(s) allocated from:
    #0 0x7fda668fe548 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:95
    #1 0x5564bcfc6901 in hazelcast::client::test::thread_pool::ThreadPoolTest_testEqualThreadAndJobs_Test::TestBody() /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/hazelcast/test/src/HazelcastTests8.cpp:2733
    #2 0x5564be3078f3 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2433
    #3 0x5564be2f8fa0 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2469
    #4 0x5564be2a3afd in testing::Test::Run() /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2508
    #5 0x5564be2a4fd0 in testing::TestInfo::Run() /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2684
    #6 0x5564be2a5d2b in testing::TestSuite::Run() /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2816
    #7 0x5564be2c28ef in testing::internal::UnitTestImpl::RunAllTests() /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:5338
    #8 0x5564be30aca8 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2433
    #9 0x5564be2fb991 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2469
    #10 0x5564be2bf3ac in testing::UnitTest::Run() /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:4925
    #11 0x5564be3291f0 in RUN_ALL_TESTS() /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/include/gtest/gtest.h:2473
    #12 0x5564be32913c in main /home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest_main.cc:45
    #13 0x7fda6602a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #14 0x7fda6602a28a in __libc_start_main_impl ../csu/libc-start.c:360
    #15 0x5564bc4f0e14 in _start (/home/runner/work/hazelcast-cpp-client/hazelcast-cpp-client/build/hazelcast/test/src/client_test+0x3e2e14) (BuildId: 1ae11315358a2aa750dab7fc7ac35241e01a2005

...the bad news is that these failures are actually blocking this PR from being merged.

What do you think the next steps should be?

  • fix the leaks separately (^ that ^ one looks like it's in test code, there are probably others)
  • try suppressing leaks checking and aim for stacktraces only

@JackPGreen JackPGreen requested a review from ihsandemir May 16, 2025 22:27
@JackPGreen JackPGreen marked this pull request as ready for review May 16, 2025 22:27
@JackPGreen JackPGreen force-pushed the add-`AddressSanitizer`-3 branch from b7cef7f to 573b430 Compare May 17, 2025 09:05
JackPGreen added a commit to JackPGreen/hazelcast-cpp-client that referenced this pull request May 17, 2025
The change in hazelcast#1279 made it obvious that adding configuration for `Debug` builds (e.g. "add an extra compilation argument") was very tedious as it needs to be declared in _each_ workflow and then passed all the way down to the actual scripts that build the code.

Instead, it'd be much cleaner if the jobs _only_ declare a release type, and the build scripts react to that with the appropriate actions (e.g. adding some extra debug or whatever).

Changes:
- migrate the _implementation_ of the `DEBUG` builds into the `build-` scripts
   - e.g. `WARN_AS_ERROR` can be assumed when `DEBUG` is enabled, rather than being duplicated in each job configuration
   - remove the architecture to pass these redundant params down
- pass `BUILD_TYPE` into the `build-` scripts as an environment variable, as used for other parameters
-  rename `BUILD_CONFIGURATION` to `BUILD_TYPE` in Windows scripts for consistency
- enable `WARN_AS_ERROR` on Windows for consistency

This PR also helps to make hazelcast#1279 smaller.
JackPGreen added a commit to JackPGreen/hazelcast-cpp-client that referenced this pull request May 17, 2025
The change in hazelcast#1279 made it obvious that adding configuration for `Debug` builds (e.g. "add an extra compilation argument") was very tedious as it needs to be declared in _each_ workflow and then passed all the way down to the actual scripts that build the code.

Instead, it'd be much cleaner if the jobs _only_ declare a release type, and the build scripts react to that with the appropriate actions (e.g. adding some extra debug or whatever).

Changes:
- migrate the _implementation_ of the `DEBUG` builds into the `build-` scripts
   - e.g. `WARN_AS_ERROR` can be assumed when `DEBUG` is enabled, rather than being duplicated in each job configuration
   - remove the architecture to pass these redundant params down
- pass `BUILD_TYPE` into the `build-` scripts as an environment variable, as used for other parameters
-  rename `BUILD_CONFIGURATION` to `BUILD_TYPE` in Windows scripts for consistency
- enable `WARN_AS_ERROR` on Windows for consistency

This PR also helps to make hazelcast#1279 smaller.
@JackPGreen JackPGreen force-pushed the add-`AddressSanitizer`-3 branch 3 times, most recently from 7dfca18 to 7d23115 Compare May 17, 2025 20:50
@JackPGreen JackPGreen force-pushed the add-`AddressSanitizer`-3 branch from 7d23115 to e1a6f1e Compare May 17, 2025 20:51
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants