Benchmark

The purpose of this repository is to provide some C++ micro-benchmarks on common datastructures, to help us decide which one to use in which cases.

All the micro-benchmarks are done using google-benchmark

Hashmap

Lot's of hashmap implementations are available and in proprietary code you can often find custom implementations (for the best or worst) hashmap.x.cpp benchmarks right now

• std::unordered_map
• absl::unordered_map
• boost::unordered_map
• QHash
• ska::flat_hash_map
• tsl::robin_map
• folly::F14FastMap

Here is an example taken from the generated output for the last benchmarks. It measure the time it takes to randomly insert and after few other operations (insert and erase) erase the element.

Requirements

The repository is using submodules for thirparties libraries so make sure you also clone the submodules before running cmake

Run the benchmarks

The project is using CMake build-system, here is an example to configure, build and run the benchmarks

$ mkdir -p build
$ cmake -DCMAKE_BUILD_TYPE=Release -DBENCHMARK_ENABLE_TESTING=OFF . build
$ cd build && make -j12
$ cd -

Once everything is build you can just run the benchmark and wait for the results to come! You will need python3 and jinja2 package installed to generate the plots

$ build/yoshi/hashmap/hashmap --benchmark_min_time=2 --benchmark_format=json | python3 benchmark.py

Once this is done this will open a web browser with all the plots in one page.

Compare 2 implementations

You can also run the benchmark more precisely to compare for example only 2 implementations, that line will only run the short mode

$ build/yoshi/hashmap/hashmap -s --benchmark_filter=".*absl.*|.*folly.*"

If you just want to run one test case for two implementations you can be even more precise in the regexp

$ ./hashmap -s --benchmark_filter="Insert_Erase_Random.*absl.*|Insert_Erase_Random.*folly.*"

TODO

• try to make benchmark.py more generic
  • it only works for cases called with BENCHMARK_TEMPLATE()->Arg
  • it should support several input files
• add benchmarks for other containers