Heap use after free for file scope globals

[markcmiller86]

Mark,

I’m running Ardra through the clang sanitizer right now, and I got the error message below. This is the first time I’ve tried Silo 4.11.0. But it seems like some memory is freed then used all within one of our calls to silo.

I’m happy to help with more information, but I’m not sure what you might need. Or if it is even your fault!

Tom

0: =================================================================
0: ==6095==ERROR: AddressSanitizer: heap-use-after-free on address 0x61d0000460b8 at pc 0x00000d8ebaee bp 0x7fffffff4d50 sp 0x7fffffff4d48
0: READ of size 8 at 0x61d0000460b8 thread T0
0: #0 0xd8ebaed in DBClose /tmp/brunner6/spack-stage/spack-stage-silo-4.11.0-yg6rktobvy7g247oep4ztxygc2o6m2fv/spack-src/src/silo/silo.c:4426
0: #1 0x234ada2 in Ardra::IO::DBHandleSilo::~DBHandleSilo() /g/g22/brunner6/wci/ardra/transport3d/Ardra/IO/DBHandleSilo.cpp:55
0: #2 0x234b23f in Ardra::IO::DBHandleSilo::~DBHandleSilo() /g/g22/brunner6/wci/ardra/transport3d/Ardra/IO/DBHandleSilo.cpp:61
0: #3 0x23bd174 in Ardra::IO::DumpWriter::writeDump(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/IO/DumpWriter.cpp:92
0: LLNL/miller86-issue-mover-tmp#1 0x19b6506 in Ardra::Process::ArdraProcess::writeRestart(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:452
0: #5 0x19b2261 in operator() /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:398
0: LLNL/miller86-issue-mover-tmp#2 0x19beb13 in _M_invoke /usr/tce/packages/gcc/gcc-8.1.0/include/c++/8.1.0/bits/std_function.h:297
0: #7 0x207e274 in std::function<void ()>::operator()() const /usr/tce/packages/gcc/gcc-8.1.0/include/c++/8.1.0/bits/std_function.h:687
0: #8 0x230ed5a in Ardra::Driver::SteadyStateDriver::run(Armus::Domain::ComputeData&, Armus::Options::Options&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Driver/SteadyStateDriver.cpp:160
0: #9 0x19b3b60 in Ardra::Process::ArdraProcess::runDriver(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:404
0: #10 0x11e19c9 in Ardra::API::ArdraAPI::runCycle(double, double, void*) /g/g22/brunner6/wci/ardra/transport3d/Ardra/API/ArdraAPI.cpp:409
0: #11 0x1a5fdd8 in ardra_main(Ardra::API::ArdraAPI&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/ardra_main.cpp:134
0: #12 0x11bda63 in main2 /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/main.cpp:49
0: #13 0x11bf5f0 in main /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/main.cpp:122
0: #14 0x2aaaaf40f554 in __libc_start_main ../csu/libc-start.c:266
0: #15 0x11bc148 (/usr/WS1/brunner6/ardra/ardra-toss_3_x86_64_ib/ardra+0x11bc148)
0:
0: 0x61d0000460b8 is located 56 bytes inside of 2040-byte region [0x61d000046080,0x61d000046878)
0: freed by thread T0 here:
0: #0 0x2aaaaadba2c0 in __interceptor_free /builddir/build/BUILD/gccspack/spack/var/spack/stage/gcc-8.1.0-yf4dn5leietjepntgrnkv4syhgmb2nmm/gcc-8.1.0/libsanitizer/asan/asan_malloc_linux.cc:66
0: #1 0xd8bcfe7 in silo_db_close /tmp/brunner6/spack-stage/spack-stage-silo-4.11.0-yg6rktobvy7g247oep4ztxygc2o6m2fv/spack-src/src/silo/silo.c:762
0: #2 0xda72dbe in db_hdf5_Close /tmp/brunner6/spack-stage/spack-stage-silo-4.11.0-yg6rktobvy7g247oep4ztxygc2o6m2fv/spack-src/src/hdf5_drv/silo_hdf5.c:6058
0: #3 0xd8eb9b5 in DBClose /tmp/brunner6/spack-stage/spack-stage-silo-4.11.0-yg6rktobvy7g247oep4ztxygc2o6m2fv/spack-src/src/silo/silo.c:4425
0: LLNL/miller86-issue-mover-tmp#1 0x234ada2 in Ardra::IO::DBHandleSilo::~DBHandleSilo() /g/g22/brunner6/wci/ardra/transport3d/Ardra/IO/DBHandleSilo.cpp:55
0: #5 0x234b23f in Ardra::IO::DBHandleSilo::~DBHandleSilo() /g/g22/brunner6/wci/ardra/transport3d/Ardra/IO/DBHandleSilo.cpp:61
0: LLNL/miller86-issue-mover-tmp#2 0x23bd174 in Ardra::IO::DumpWriter::writeDump(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/IO/DumpWriter.cpp:92
0: #7 0x19b6506 in Ardra::Process::ArdraProcess::writeRestart(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:452
0: #8 0x19b2261 in operator() /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:398
0: #9 0x19beb13 in _M_invoke /usr/tce/packages/gcc/gcc-8.1.0/include/c++/8.1.0/bits/std_function.h:297
0: #10 0x207e274 in std::function<void ()>::operator()() const /usr/tce/packages/gcc/gcc-8.1.0/include/c++/8.1.0/bits/std_function.h:687
0: #11 0x230ed5a in Ardra::Driver::SteadyStateDriver::run(Armus::Domain::ComputeData&, Armus::Options::Options&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Driver/SteadyStateDriver.cpp:160
0: #12 0x19b3b60 in Ardra::Process::ArdraProcess::runDriver(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:404
0: #13 0x11e19c9 in Ardra::API::ArdraAPI::runCycle(double, double, void*) /g/g22/brunner6/wci/ardra/transport3d/Ardra/API/ArdraAPI.cpp:409
0: #14 0x1a5fdd8 in ardra_main(Ardra::API::ArdraAPI&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/ardra_main.cpp:134
0: #15 0x11bda63 in main2 /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/main.cpp:49
0: #16 0x11bf5f0 in main /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/main.cpp:122
0: #17 0x2aaaaf40f554 in __libc_start_main ../csu/libc-start.c:266
0:
0: previously allocated by thread T0 here:
0: #0 0x2aaaaadba828 in __interceptor_calloc /builddir/build/BUILD/gccspack/spack/var/spack/stage/gcc-8.1.0-yf4dn5leietjepntgrnkv4syhgmb2nmm/gcc-8.1.0/libsanitizer/asan/asan_malloc_linux.cc:95
0: #1 0xdadf849 in db_hdf5_Create /tmp/brunner6/spack-stage/spack-stage-silo-4.11.0-yg6rktobvy7g247oep4ztxygc2o6m2fv/spack-src/src/hdf5_drv/silo_hdf5.c:5995
0: #2 0xd949f76 in DBCreateReal /tmp/brunner6/spack-stage/spack-stage-silo-4.11.0-yg6rktobvy7g247oep4ztxygc2o6m2fv/spack-src/src/silo/silo.c:4326
0: #3 0x2342dbb in Ardra::IO::DBFactorySilo::createWriteHandle(unsigned long) /g/g22/brunner6/wci/ardra/transport3d/Ardra/IO/DBFactorySilo.cpp:77
0: LLNL/miller86-issue-mover-tmp#1 0x23b9e9a in Ardra::IO::DumpWriter::writeDump(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/IO/DumpWriter.cpp:39
0: #5 0x19b6506 in Ardra::Process::ArdraProcess::writeRestart(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:452
0: LLNL/miller86-issue-mover-tmp#2 0x19b2261 in operator() /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:398
0: #7 0x19beb13 in _M_invoke /usr/tce/packages/gcc/gcc-8.1.0/include/c++/8.1.0/bits/std_function.h:297
0: #8 0x207e274 in std::function<void ()>::operator()() const /usr/tce/packages/gcc/gcc-8.1.0/include/c++/8.1.0/bits/std_function.h:687
0: #9 0x230ed5a in Ardra::Driver::SteadyStateDriver::run(Armus::Domain::ComputeData&, Armus::Options::Options&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Driver/SteadyStateDriver.cpp:160
0: #10 0x19b3b60 in Ardra::Process::ArdraProcess::runDriver(Armus::Domain::ProcessData&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/Process/ArdraProcess.cpp:404
0: #11 0x11e19c9 in Ardra::API::ArdraAPI::runCycle(double, double, void*) /g/g22/brunner6/wci/ardra/transport3d/Ardra/API/ArdraAPI.cpp:409
0: #12 0x1a5fdd8 in ardra_main(Ardra::API::ArdraAPI&) /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/ardra_main.cpp:134
0: #13 0x11bda63 in main2 /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/main.cpp:49
0: #14 0x11bf5f0 in main /g/g22/brunner6/wci/ardra/transport3d/Ardra/boltzmann3d/main.cpp:122
0: #15 0x2aaaaf40f554 in __libc_start_main ../csu/libc-start.c:266

[markcmiller86]

This is happening here,

Silo/src/silo/silo.c

Line 4450 in 014c31a

free(dbfile->pub.file_scope_globals);

This line should be moved into db_silo_close() so that the memory pointed to by pub.file_scope_globals is freed before the memory holding that pointer is freed.

[markcmiller86]

Ok, what is happening here is that the dbfile pointer is being freed before a separately allocated member that the dbfile pointer points to is freed. Hence, when file_scope_globals is freed, the pointer holding it was already freed and we're attempt to access freed heep memory.

[markcmiller86]

Resolved on 4.11RC in #306
Resolved on main in #326