Uploading a new version of a Data Package

(e.g. datapackage_renewable_power_plants)

0. Merging branch development with master (optional)

If you have worked in the development branch instead of the master branch, you should first merge the development branch with the master branch before you upload a new version.

• Add and commit all changes you want to keep in the development branch and push it to the remote repository (github)
  > git add 'filename'
> git commit -m "commit message"
• Merge branches locally > git checkout development (to be sure to be on branch development)
  > git merge master (on branch development)

(resolve merge conflicts if there are any)
> git checkout master
> git merge development (there won't be any conflicts now)

1. Preparation

• Several changes in script and/or files worth a new version (e.g. updated original data, new features, etc.)
• Note in the .json defining part in the script:
  changes
  date of version (e.g. 2016-06-07)
  date in the link to the tagged version (e.g. https://github.com/Open-Power-System-Data/datapackage_renewable_power_plants/blob/2016-06-07/main.ipynb)
• If there is the option to download original data from the original data folder of the OPSD site, change adapt the version in the code (e.g. http://data.open-power-system-data.org/datapackage_renewables/2016-07-11/original_data
• Check if the requirements.yml is still up to date and includes all but just the packages required for the scripts in the Data Package
  conda env export > requirements.yml
• Run script(s)
• Check output files (csv, xlsx, json)
• Clear output in script

2. Upload files

• Create folder with the name of the version (e.g. 2016-06-07)
• Within this folder, create the folder: 'original_data'. Put the original data for exactly this version into the folder. (e.g. folders 'BNetzA' and 'Netztransparenz' with the respective zip, csv and excel files)
• Add the output files (json/csv/xlsc/sqlite) directly into the version folder, e.g. for renewables Data Package:
  datapackage.json renewable_capacity_germany_timeseries.csv
  renewable_power_plants_germany.csv
  renewable_power_plants_germany.sqlite
  renewable_power_plants_germany.xlsx
  validation_marker.csv
• Connect via FTP to OPSD data server (see Dropbox/Datenplattform/Daten/FTPZugangsdaten.txt for Host/User/PW)
• Check if the new folder has the same structure as the folders of the older versions
• Upload the folder with the version name (e.g. 2016-06-07)
• Create zip-folder for this version on the page
• If changes made to the data constitute a change in the data availability, create an updated copy of availabilities.json and send it to Jonathan.

3. Update scripts on github and create a version tag

• Either delete output folder / original_data / sqlite-file etc. or add all the files that should not be included in the git version control in the file .gitignore
• Go into the git-folder (e.g. < cd Documents/opsd/datapackage_renewable_power_plants)
• Push branch master in remote repository (GitHub)
  > git checkout master (to make sure you are on the master branch)
  > git push origin master (push to the remote repository)
• create a tag with the respective version name, e.g. 2016-06-07
  > git tag -a 2016-07-11 -m "2016-07-11" (create tag locally)
  > git push origin 2016-07-11 (push the tag to the remote repository)
• Check on GitHub if on the master branch there is the actual code and if the tag works correctly (On the left, where you can find the branches, you can also find the tags)