# AidData's Geo Framework
Core components of AidData's Geo Framework running on William and Mary's SciClone High Permance Computing cluster.

http://geo.aiddata.wm.edu

## Setup

### core hpc config
- make sure HPC account being used is set as priority user on for vortex-alpha nodes (HPC staff can do this)
- make sure HPC servers have necessary ports open for mongodb, gmail

### prepare database server
- have IT open mongodb ports for geo.aiddata.wm.edu and all HPC servers (prod and dev servers, where applicable)
- update mongod.conf
- copy db_backup_script.sh and add cron (see comments in script for details)


### prepare sciclone environment
- install home directory environment scripts (copy from `sciclone` in this repo to your sciclone account home directory, or extract from the `home_backups` dir in `/sciclone/aiddata10/geo`)
- load necessary modules by logging out and back in to sciclone or using `source ~/.cshrc` and `source ~/.cshrc.rhel6-opteron`
- install python packages: `pip install --user -r pip_list.txt`
- add ssh key from your sciclone account to `aiddatageo` github (approve rsa key first time manually)

### initialize framework
- run `bash setup.sh <branch>`


## Ingest Datasets
- utilizes data and ingest files from asdf-datasets repo
- see ingest dir for specifics on ingesting datasets (related resources in asdf-datasets repo)


## Components

### Tasks

todo


### Tools

todo


### Utils

todo


### Assets

static files/resources used by other scripts (e.g., images, text templates, pdfs)


### Scr

Scratch - misc scripts and tools not critical to running geo framework


## Tests

py-test based tests for testing geo utiltiies and components


## extract-scripts

Tools for manually running batch extract jobs


## Related Repos / Resources

Wiki - Additional details about Geo Framework and Sciclone:
- https://github.com/itpir/geo-hpc/wiki

Dataset preparation and ingest:
- https://github.com/itpir/geo-datasets

Website source and related config files (Private repos):
- https://github.com/itpir/geo-query
- https://github.com/itpir/geo-core

Related:
- https://github.com/itpir/geo-portal
- https://github.com/itpir/geoMatch
- https://github.com/itpir/geoML
- https://github.com/itpir/geoDash
- https://github.com/itpir/geoValuate

Previous repos for components of the Geo Framework which were combined in this repo:
- https://github.com/itpir/asdf
- https://github.com/itpir/mean-surface-rasters
- https://github.com/itpir/extract-scripts
- https://github.com/itpir/det-module