Skip to content

Checklist for Generating and Publishing netCDF Files

Steve Smith edited this page Jun 21, 2018 · 6 revisions

Checklist before generating new netCDF files

  • If a new dataset is being created, then the version number should be incremented. This currently a manual setting inside the file code/parameters/global_settings.R.
  • Ensure global location variable matches where the system is being run from.
  • Ensure global voc_speciation variable is set correctly
  • Unless intentionally debugging, make sure the debug flag in exe/launch/launch_downscale_gridding.R is set to FALSE

Checklist before using netCDF files

Users should check a number of items before using or distributing netCDF files produced by this software.

  1. Check that the system ran and that new netCDF files were written to the final output directory you specified (often final-output/module-C/). The creation date can also be found in the metadata's history variable.
  2. Look for any error files in diagnostic-output/. These files are of the form ERROR_MM-DD-HHmmSS, where MM-DD-HHmmSS is the month, day, hour, minute, and second that the run started. Rows are written to the error files if any scenarios, emissions, and sectors have changed values beyond the allowed threshold (set by the global parameter error_tolerance).
  3. Confirm that global totals are equal before and after running the gridding routine. The directory diagnostic-output/ contains difference files (these files end in _DIFF), a sample of which should be checked to assure that the original global totals match the gridded global totals.
  4. Visually check emission maps to verify that the sector order, where this is relevant, matches the metadata
  5. Visually check a sample of maps for unexpected hot spots or data holes.
  6. Check the diagnostic line graphs produced by the system for unexpected discontinuities or other issues. These are located in the diagnostic-output/ directory and cover specific geographic areas.
  7. Metadata: Check that the names of the files, creation location, institution name, and version numbers have been set appropriately in the metadata. Tools such as Panoply or ncdump are helpful.
  8. Metadata pt2: As part of checking metadata it is useful, for each file type, to check a complete set of metadata against a previously generated sample that is known to be correct (most useful to do this using a visual text difference application).

Visual confirmation of shipping emissions using Panoply:

Panoply example