Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

alacarte-importer uses excessive amounts of memory #48

Open
Venemo opened this issue Feb 14, 2015 · 2 comments
Open

alacarte-importer uses excessive amounts of memory #48

Venemo opened this issue Feb 14, 2015 · 2 comments

Comments

@Venemo
Copy link

Venemo commented Feb 14, 2015

Seems that alacarte-importer actually allocates a huge amount of memory when processing the input file and at the end throws an std::bad_alloc.

Here is an example output:

pi@raspberrypi ~/alacarte/build-rpi $ ./alacarte-importer ~/hungary-latest.osm data.carte
[INFO] [] Logfile opened...
[INFO] [Configuration] config: /etc/alacarte.conf
[INFO] [Configuration] Unknown type[b]
[INFO] [Configuration] importer.geo-data: data.carte
[INFO] [Configuration] importer.osm-data: /home/pi/hungary-latest.osm
[INFO] [Configuration] logfile: log.txt
[INFO] [] Start parsing...
[INFO] [Importer] Load xml-file "/home/pi/hungary-latest.osm"
[INFO] [Importer] File size is 1503232kb
[INFO] [Importer] Bounds tag in osm data is ignored by this software!
[INFO] [Importer] Loading [1%]
[INFO] [Importer] Loading [2%]
[INFO] [Importer] Loading [3%]
[INFO] [Importer] Loading [4%]
[INFO] [Importer] Loading [5%]
[INFO] [Importer] Loading [6%]
[INFO] [Importer] Loading [7%]
[INFO] [Importer] Loading [8%]
[INFO] [Importer] Loading [9%]
[INFO] [Importer] Loading [10%]
[INFO] [Importer] Loading [11%]
[INFO] [Importer] Loading [12%]
[INFO] [Importer] Loading [13%]
[INFO] [Importer] Loading [14%]
[INFO] [Importer] Loading [15%]
[INFO] [Importer] Loading [16%]
[INFO] [Importer] Loading [17%]
[INFO] [Importer] Loading [18%]
[INFO] [Importer] Loading [19%]
[INFO] [Importer] Loading [20%]
[CRIT] [] Boost diagnostic:
Dynamic exception type: std::bad_alloc
std::exception::what: std::bad_alloc
@TheMarex
Copy link
Contributor

Yeah, the code that handles the import does not really scale to files of that size. :-)
What should be used instead of hand-rolled code is something like http://osmcode.org/libosmium/

@Venemo
Copy link
Author

Venemo commented Feb 15, 2015

The code isn't bad at all. It imports that file on my laptop in a few minutes. However on the Raspberry Pi it took it 30 minutes to go to 20% and crash. Now I've added some swap to the Raspberry Pi and (due to excessive swapping) it took it ~9 hours to reach 99%.

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

No branches or pull requests

3 participants