This project is used to crawl & compile websites and data sources to generate accurate place information. All data can be found in the data/
directory. All spiders (used to crawl an individal source) can be found in the spiders/
directory.
Each output of data is stored in a GeoJSON file as a FeatureCollection. Each root object of the GeoJSON file will also contain an updated_at
property, which is a number representing the UNIX timestamp (in miliseconds) that the file was updated at.
Although the output data mostly conforms to the GeoJSON standard, in the event that coordinates can not be found for a given spider, the geometry
object will be null
.
Most feature properties
tags will conform to OpenStreetMap's tagging system.
To run all spiders you can run:
$ node placecompile crawl
To run a specific spider you can run:
$ node placecompile crawl SPIDERNAME
Replacing SPIDERNAME
with the name of the spider file you wish to run (ex. amctheatres
).
Please feel free to submit pull requests to help improve the project, or issues if you notice a problem or have a bug to report.
The software and data is released under the Unlicense.