A simplistic GDB pretty printer for nlohmann-json c++

Provides GDB script and python GDB pretty printer script that allows to to print a nlohmann / json

• compatible with a live inferior process with debug symbols
• compatible with core dump files with debug symbols

This is also a playground for me to get used to git, Github, gitflow and GDB python scripting/pretty printing.

Release notes:

v0.0.1: first pretty printer release

Features:

• improved overall GDB python pretty printer code
  • now multiplatform (verified on some)
• created some sort of a CI process to check we did not mess up with features:
  • checks that the pretty printer output matches the json.dump() output.
  • checks various GDB releases + python versions on Ubuntu
  • also checks on windows server (but only the gnat community GDB version obtained with chocolatey)

---

Table of contents

1. Prerequisites
2. Installing
3. Content
4. Usage
5. Possible improvements / Contributions
6. Known limitations
7. Examples and tests
8. History
9. Acknowledgments / LICENSES
10. Links concerning STL and GDB

1. Prerequisites

• GDB debugger installed, ready to use, and of one of the versions below

  • Tested on Ubuntu x64, with both python 2.7 and python 3.6:
    • GDB 7.12.1
    • GDB 8.0
    • GDB 8.0.1
    • GDB 8.1
    • GDB 8.1.1
    • GDB 8.2
    • GDB 8.2.1
    • GDB 8.3
    • GDB 8.3.1
    • GDB 9.1
  • Windows
    • Server 2019 (win 10; x86_64) and Windows 10 Pro x86_64 GDB 8.3 with python 2.7.10 (from GNAT CE 2019)
    • Given the successful tests on Ubuntu x64 with various GDB and python versions, it is likely to work for the GDB + python versions above on Windows too.
  • Tested on Raspbian arm 32, with python 2.7 and GDB 8.3.1
    • Given the successful tests on Ubuntu x64 with various GDB and python versions, it is likely to work for the GDB versions above on Windows too.

• an executable to debug with debug symbols available to GDB which uses the JSON lib 3.7.3. No other versions tested yet.

• or a core dump with debug symbols available to GDB (for linux users)

• Some GDB commands knowledge might be useful for your debug session to be successful

Your GDB does not support python ?

You need to upgrade your GDB.

Have a look on this wiki page for an example of GDB build on raspbian 9.11

Optional

• a GNAT CE 2019 install to compile and play with the provided test projects

2. Installing

Just copy the GDB and/or python script you need in a folder near your executable to debug, and of course, load it into your GDB. For linux users, you can do a wget on the file (or use the release package, decompress, and use the file you want)

# get the file
$ wget https://raw.githubusercontent.com/LoneWanderer-GH/nlohmann-json-gdb/master/scripts/nlohmann_json.gdb
# start GDB session
$ gdb
(gdb) file ... # load your exe
(gdb) source nlohmann_json.gdb
# print a JSON variable
(gdb)pjson foo
{
    "flex" : 0.2,
    "awesome_str": "bleh",
    "nested": {
        "bar": "barz"
    }
}

or

# get the file
$ wget https://raw.githubusercontent.com/LoneWanderer-GH/nlohmann-json-gdb/master/scripts/nlohmann_json.gpy
# start GDB session
$ gdb
(gdb) file ... # load your exe
(gdb) source nlohmann_json.py
# print a JSON variable
(gdb)p foo
$ 1 = {
    "flex" : 0.2,
    "awesome_str": "bleh",
    "nested": {
        "bar": "barz"
    }
}

For windows users, its basically the same except you may not be able to download the file in command line, links are provided in Content below. Also, your GDB might be embedded in some IDE, but its most likely a GDB console front-end.

See also Content and Usage sections below for more details of what you may find in this repo.

3. Content

• the GDB command file : it uses the live process under debug to call dump(). It implies that the executable and memory are not corrupted, variables not optimized out
• the GDB python pretty printer file : here, we do not rely on the existing dump() method but we explore memory using debug symbols to do it ourselves, if the inferior process is broken in some way, we may still have some means to dump a json compare to previous method.

Additional content:

• some tests projects, see 7. Examples / Tests for further details:
  • a c++ test project
  • a c++ test project to bruteforcefully find relevant offets for a given platform

4. Usage

How to load a GDB script

in your GDB console:

(gdb) source some_file

Works for both GDB and Python scripts. I strongly suggest you refer to GDB documentation.

GDB pretty printer usage (the Python printer)

The GDB Pretty printer is written in Python.

To load it into GDB (you may adapt path to your settings):

(gdb) source nlohmann_json.py

Then, a simple GDB command does the trick:

(gdb) p foo
$ 1 = {
    "flex" : 0.2,
    "awesome_str": "bleh",
    "nested": {
        "bar": "barz"
    }
}

GDB script usage (the GDB command)

Here we use a kind of GDB macro defined in a GDB script file

To load it into GDB (you may adapt path to your settings):

(gdb) source nlohmann_json.gdb

(gdb) pjson foo
{
    "flex" : 0.2,
    "awesome_str": "bleh",
    "nested": {
        "bar": "barz"
    }
}

notice that pjson does not print the GDB history tag $

No debug symbols ?

That's a more advanced GDB technique. You should have a look at this SO post where pretty much everything is explained.

The idea is to compile and extract the debug data into specific files. Then load this files into your GDB to have all symbols at hand, even if you're working with a stripped software.

see also this GDB doc concerning symbol-file command.

5. Possible improvements / Contributions

Contribute

Coding technique for the pretty printer is quite naive, but it works. Any seasoned advice and support appreciated. Aspects I would like to improve:

• performance
• code style
• Release packaging
• Lib version checks

Possible TODO list

• dont use this TODO list, but Github issues and Github project management

• printer can be customised further to print the 0x addresses, I chose not to since the whole point for me was NOT to explore them in GDB. You would have to add few python print here and there

• add the hexa value for floating point numbers, or for all numerical values

• reduce amount of copy/pasta between offsets_finder.py and nlohmann_json.py

• the pythonGDBpretty printer core dump management is not (yet ?) done (i.e. core dump means no inferior process to call dump() in any way, and possibly less/no (debug) symbols to rely on) Core dump with debug symbols tested and should be working.

• Improve method to get std::string type and sizeof. The current method assumes some known symbols names, that most probably depends on the compilation tools (C++11). Sadly, GDB command whatis and ptype cannot resolve directly and easily std::string Solved with the gdb type template argument type extraction feature

6. Known limitations

• Floating point numbers may appear differently depending on the method used. This is due to differences in float-to-string from GDB and json c++. For more confidence, we could modify the python pretty printer to provide the exact hexadecimal memory value + the decimal one for sake of completness. However, the checks using python json module show no difference concerning floats once parsed.

• Linux over windows (Ubuntun-windows) : gprbuild command on Ubuntu-windows/Debian-windows may not work correctly, so a legit Linux environment may be needed if you want to play with the tests projects on Linux.

7. Examples and tests

Sample C++ project

The C++ project is located in tests/cpp_test_project

1. Build debug_printer.gpr with the following command

   cd tests/cpp_test_project
   gprbuild -p
   

   (-pcreates the obj/exe dirs if missing; gpr file can be deduced from current folder content)

2. see main.cpp for some basic C++ JSON declarations. It should looke like:

   // C++ code
   ...
   json foo;
   foo["flex"] = 0.2;
   foo["bool"] = true;
   foo["int"] = 5;
   foo["float"] = 5.22;
   foo["trap "] = "you fell";
   foo["awesome_str"] = "bleh";
   foo["nested"] = {{"bar", "barz"}};
   foo["array"] = { 1, 0, 2 };
   ...
   

3. Once the exe is built, launch a GDB session, either in console or using your favorite IDE.

   2 cases :

   • GDB autoloads the .gdbinit file located near the gpr file
   • GDb does not autoload the .gdbinit file. In this case type in gdb:

     (gdb) source .gdbinit
     

4. Now you can use the following GDB commands:

   (gdb) pjson foo
   {
       "array": [
           1,
           0,
           2
       ],
       "awesome_str": "bleh",
       "bool": true,
       "flex": 0.2,
       "float": 5.22,
       "int": 5,
       "nested": {
           "bar": "barz"
       },
       "trap ": "you fell"
   }
   

   GDB python pretty printer:

    (gdb) p foo
    {
       "array" : [
               1,
               0,
               2,
       ],
       "awesome_str" : "bleh",
       "bool" : true,
       "flex" : 0.20000000000000001,
       "float" : 5.2199999999999998,
       "int" : 5,
       "nested" : {
               "bar" : "barz"
       },
       "trap " : "you fell",
   }
   

C++ project: awful bruteforce method to check that memory offsets are correct

This part will tell if the method to find data offsets in the proposed python script is correct.

1. build the project simple_offsets_finder.gpr with the command gprbuild -p -P debug_printer.gpr

2. Start a GDB session, using this console command to launch GDB. It should tell you if the GDB code deduced offset values are consistent with the bruteforce approach.

   gdb --se=exe/main.exe -command=find_offsets.gdb --batch
   

Another approach I know of

from a guru of my workplace

• simply define a single function in the program to perform the dump of a json variable, say print(). Then you can call it during yourGDBsession. This is almost exactly similar to theGDBinferior dump() call macro pjson presented above.

8. History

• In March 2019, I was stuck with the lack of nlohmann json debug utilities. I could not find any support to print what the json was during a debug session. I ended up with a stack overflow post with what I found to be revelant for that matter. In addition, I was interested in playing around with GDB/memory/python, thats why I took some time to treat this matter. I ended up with the code here.
• In 2020 a Github issue was opened with the same intent here and relies basically on the same initial solution I used: nlohmann/json#1952

I'm not claiming any right or precedence over the official nlohmann / json issue or the method to perform the print using dump(). I think we all did the same thing by serendipity, and I bet I am not the first one to have taken the .dump() call approach. All in all, if everyone can work debug better, that all that matters to me

9. Acknowledgments / LICENSES

ACKNOWLEDGMENTS

• The GDB documentation was particularly useful, in particular the Python part.
• The red black tree traversal and STL exploration in python is directly inspired from the STLGDBscripts.

A few other links were useful and are linked in the python source.

• The python exceptions printing is inspired from GEF
• The std::string print in pythonGDBis inspired from this stackoverflow post

LICENSE

My work is under following license

Licensed under the MIT License http://opensource.org/licenses/MIT. SPDX-License-Identifier: MIT

Copyright (c) 2020 LoneWanderer-GH https://github.com/LoneWanderer-GH

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Some files in this repo are not mine and under other licenses

nlohmann JSON for Modern C++

as per the file content:

JSON for Modern C++ version 3.7.3 https://github.com/nlohmann/json

Licensed under the MIT License http://opensource.org/licenses/MIT SPDX-License-Identifier: MIT Copyright (c) 2013-2019 Niels Lohmann http://nlohmann.me

STL GDB evaluators/views/utilities - 1.03

as per the file content:

Simple GDB Macros writen by Dan Marinescu (H-PhD) - License GPL Inspired by intial work of Tom Malnar, Tony Novac (PhD) / Cornell / Stanford, Gilad Mishne (PhD) and Many Many Others. Contact: dan_c_marinescu@yahoo.com (Subject: STL)

Modified to work with g++ 4.3 by Anders Elton Also added _member functions, that instead of printing the entire class in map, prints a member.

10. Links

Some useful links concerning STL and GDB

• https://sourceware.org/gdb/wiki/STLSupport
• http://www.yolinux.com/TUTORIALS/src/dbinit_stl_views-1.03.txt
• http://wiki.codeblocks.org/index.php/Pretty_Printers
• https://github.com/NREL/EnergyPlus/wiki/Debugging-STL-objects-in-GDB
• https://stackoverflow.com/questions/11606048/how-to-pretty-print-stl-containers-in-gdb
• https://gist.github.com/chaozh/9252fc01b3723f795589
• https://gist.github.com/skyscribe/3978082