Demo Shard Import Instructions

To minimize the bus factor until we release V2, here are the brief instructions for data import on demo:

1. Build yggdrasil/assembly from the release/1.x-maint branch

2. Run the YggUtils import command on the input JSON to generate the LevelDB files. You'll need a temporary directory where the tool can save the generated files; in our example it's tmpdata:

   java -cp yggdrasil/target/yggdrasil-assembly-1.1.3-SNAPSHOT.jar com.precog.yggdrasil.util.YggUtils import -t 1BF2FA96-8817-4C98-8BCB-BEC6E86CB3C2 -s tmpdata '/summer_games/athletes=/Users/dchenbecker/Downloads/OlympicsJSON' Where:

3. -t = Token to import under

4. -s = the directory to generate the projections under (tmpdata in our example)

5. '/<path>=<input JSON file>', and you can do multiple files/paths in one run of the tool, e.g. … '/foo=foo.json' '/bar=bar.json'

6. Copy the generated directories over to qclus-demo01.reportgrid.com under the ubuntu account (substitute your temp directory for tmpdata):

   rsync -vaz tmpdata qclus-demo01.reportgrid.com:

7. SSH into qclus-demo01.reportgrid.com, then copy the new data into the demo shards:

   for sid in `seq 1 4`; do sudo rsync -vaz <new directory>/* ~ubuntu/envs/demo/shard/shard0${sid}/data/data/; done

8. Restart the demo shard:

   sudo restart shard-demo

   for sid in `seq 2 4`; do sudo restart shard-demo$sid; done