Skip to content

Access Data

Jump to bottom
davis278 edited this page Sep 3, 2019 · 17 revisions

VCDAT can read data from the following file types (which CDMS2 will also read):

If you already have data stored on NERSC Cori or ACME1 your own computer or local network, you can use that data, but it might be worth testing out VCDAT using the sample data below.

Download the Sample Data

To download some test data we've provided, enter the code below within a cell in a Jupyter notebook (like the untitled notebook you opened on the User Interface page) and run the cell by holding down shift and pressing enter while that cell is highlighted:

import vcs
import cdms2
import cdat_info
import pkg_resources
vcs_egg_path = pkg_resources.resource_filename(pkg_resources.Requirement.parse("vcs"), "share/vcs")
path = vcs_egg_path+'/sample_files.txt'
cdat_info.download_sample_data_files(path,"sample_data")

It might take a few moments for the commands to run.

The data will be stored within a subfolder called "sample_data" in the same location from which you launched your JupyterLab interface. If you look at the left panel, you should see a folder titled "sample_data". If you double click on the sample_data folder, you will see many test files in a variety of formats.

Using Your Own Data

Ensure that your NetCDF data files are located in the same directory from which you launched the JupyterLab interface or in a subdirectory (like sample_data). Once your own data files are in this directory, they should automatically show up in the JupyterLab interface and be listed in the left panel. If you do not see them immediately, refresh the browser. If you still do not see the files, click on the grey file folder in the top left corner of the interface to make sure the list of files is showing.

If you need to access data outside of your top directory, you can create a symbolic link to them.

For example assuming you would like to access data in /scratch/some_data (and assuming you can access this directory), simply go to the top directory of your user space (usually $HOME) and run this command

ln -s /scratch/some_data my_link_to_some_data

Now in the jupyter interface you should see a directory called my_link_to_some_data simply click on it and you are where you want to be.

Next Step

The next step is to load variables from a data file.

Clone this wiki locally