generated from readthedocs/tutorial-template
-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #138 from UW-GDA/egagli-patch-50
add demo nb to get snotel data from github hosted csvs
- Loading branch information
Showing
1 changed file
with
319 additions
and
0 deletions.
There are no files selected for viewing
319 changes: 319 additions & 0 deletions
319
book/modules/08_Vector_TimeSeries_SNOTEL/08_SNOTEL_download_csvs.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,319 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"id": "ad0998b7-a501-4f9f-a8c4-2ef01a23fb76", | ||
"metadata": {}, | ||
"source": [ | ||
"# 08 Demo: Using snotel_ccss_stations to get SNOTEL data\n", | ||
"\n", | ||
"UW Geospatial Data Analysis \n", | ||
"CEE467/CEWA567 \n", | ||
"Eric Gagliano " | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "5106c82f-297d-46bb-a1c0-0818bfb134e3", | ||
"metadata": {}, | ||
"source": [ | ||
"**Thanks for checking out this notebook! My hope is that this repository makes it easier to retrieve daily SNOTEL and CCSS data without having to do clunky downloads and conversions. Snow depth / SWE / PRCPSA are in meters, temperatures are in celsius. The only required packages you'll need are geopandas and pandas, the rest of the imports are for applications :)**" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "b3959e4b-74c7-426f-b903-449ebfff068e", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"import pandas as pd\n", | ||
"import geopandas as gpd\n", | ||
"import numpy as np\n", | ||
"import matplotlib.pyplot as plt\n", | ||
"import datetime\n", | ||
"import tqdm\n", | ||
"import contextily as ctx" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "0ec527a0-41b1-449e-8a59-8d456073ceca", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"source": [ | ||
"## View all SNOTEL & CCSS stations\n", | ||
"- the [SNOwpack TELemetry (SNOTEL) network](https://www.nrcs.usda.gov/wps/portal/wcc/home/aboutUs/monitoringPrograms/automatedSnowMonitoring/) includes over 800 automated weather stations in the Western U.S. for mountain snowpack observation\n", | ||
"- the [CCSS program](https://water.ca.gov/Programs/Flood-Management/Flood-Data/Snow-Surveys) manages a network of 130 automated snow sensors located in the Szierra Nevada and Shasta-Trinity Mountains" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "dad815f9-c694-4772-b9e8-cdb83f0c9d69", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"source": [ | ||
"### Read the geojson stored at https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/all_stations.geojson\n", | ||
"- the daily recurring github action should regularly update the `endDate` column \n", | ||
"- set the index of the geodataframe to the code column\n", | ||
"- let's only look at sites for which we have data" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "f7c60689-178f-42a2-b122-eacee67f5bcb", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"all_stations_gdf = gpd.read_file('https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/all_stations.geojson').set_index('code')\n", | ||
"all_stations_gdf = all_stations_gdf[all_stations_gdf['csvData']==True]" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "adc6b061-823d-44c3-ae81-f6527a23aedd", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"all_stations_gdf" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "43621e35-4326-4cf9-a221-f5f8ea614cbe", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"source": [ | ||
"### Use geopandas `GeoDataFrame.explore()` on the `all_stations_gdf` geodataframe to interactively view the stations \n", | ||
"- color by network: red is SNOTEL, blue is CCSS." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "1ef6caa1-e841-4e46-8989-afb95af787b5", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"all_stations_gdf.astype(dict(beginDate=str, endDate=str)).explore(column='network',cmap='bwr')" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "aaf197e5-7fd4-4c39-8b67-315de019f343", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"source": [ | ||
"## Read a singular CSV: *Paradise, WA*\n", | ||
"- check out information about the [SNOTEL station near Mt. Rainier at Paradise, WA](https://wcc.sc.egov.usda.gov/nwcc/site?sitenum=679)\n", | ||
"- cool plots available at the [Northwest River Forecast Center website](https://www.nwrfc.noaa.gov/snow/snowplot.cgi?AFSW1)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "1cbbc606-69e5-4898-b9bc-6bfff881215a", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"source": [ | ||
"### Place a station code (which you can find in this interactive plot, or by other means) in the url: https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station_id}.csv\n", | ||
"- for SNOTEL stations, this will be of the form {unique number}_{two letter state abbreviation}_SNTL (e.g. 679_WA_SNTL). \n", | ||
"- for CCSS stations, this will be a three letter code (e.g. BLK). \n", | ||
"- use `pd.read_csv()` with `index_col='datetime'` and `parse_dates=True` so we interpret the datetime column as pandas datetime objects" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "82c6bd2e-3d27-4908-ab24-6e61772f4e8a", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"station_id = '679_WA_SNTL'\n", | ||
"paradise_snotel = pd.read_csv(f'https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station_id}.csv',index_col='datetime', parse_dates=True)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "33bb24d7-44f9-49a5-b728-312495160f50", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"paradise_snotel" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "cc267180-8371-46da-a99d-be42c5837ffc", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"source": [ | ||
"### Try a simple plot of snow depth and SWE\n", | ||
"- select the column of interest and use pandas built in `Series.plot()`" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "cf9f9160-7bbb-4424-b58c-97412e814cb9", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"f,ax=plt.subplots(figsize=(12,5))\n", | ||
"\n", | ||
"paradise_snotel['SNWD'].plot(ax=ax,label='snow depth')\n", | ||
"paradise_snotel['WTEQ'].plot(ax=ax,label='snow water equivalent')\n", | ||
"\n", | ||
"ax.set_xlim(pd.to_datetime(['2017-10-01','2018-09-30']))\n", | ||
"\n", | ||
"ax.grid()\n", | ||
"ax.legend()\n", | ||
"\n", | ||
"ax.set_xlabel('time')\n", | ||
"ax.set_ylabel('snow depth / SWE [meters]')\n", | ||
"ax.set_title('Snow depth and SWE at Paradise, WA \\n(water year 2018)')\n", | ||
"\n", | ||
"f.tight_layout()" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "72fcc194-9e9c-4000-bea1-b3a0a9520604", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"source": [ | ||
"## Read a variable from multiple CSVs by looping over a subset of the geodataframe: *CCSS stations*\n", | ||
"- the Sierra Nevada [received a historic amount of snow in 2023](https://www.nps.gov/articles/000/sien-sierranevadamonitor-spring2023.htm)\n", | ||
"- let's explore the magnitude of this season by comparing to the median snow pack" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"id": "9c4696ca-83e1-42e2-994b-679f8d18efcb", | ||
"metadata": {}, | ||
"source": [ | ||
"### As before, create a list of the stations we are interested in, loop through and add data to a dictionary with the station code as the key, then read into pandas using `pd.DataFrame.from_dict()`\n", | ||
"- create a geodataframe `ccss_stations_gdf` of only CCSS stations from `all_stations_gdf` by creating an index where network equals CCSS\n", | ||
"- loop through the CCSS stations and create a dataframe `ccss_stations_snwd_df`" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "7d817742-e0f9-43c2-91cd-bbf90ae4ea3b", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"ccss_stations_gdf = all_stations_gdf[all_stations_gdf['network']=='CCSS']" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "0603ad1a-fd87-4623-ad13-40f9296c1d92", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"ccss_stations_gdf" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "d6688f84-bb5e-45a7-8138-771d40c1cfac", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"%%time \n", | ||
"station_dict = {}\n", | ||
"\n", | ||
"for station in tqdm.tqdm(ccss_stations_gdf.index):\n", | ||
" try:\n", | ||
" tmp = pd.read_csv(f'https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station}.csv',index_col='datetime',parse_dates=True)['SNWD']\n", | ||
" station_dict[station] = tmp\n", | ||
" except:\n", | ||
" print(f'failed to retrieve {station}')\n", | ||
"\n", | ||
"ccss_stations_snwd_df = pd.DataFrame.from_dict(station_dict).dropna(how='all')" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "0c4a5224-3312-4e8e-8582-484d5049acca", | ||
"metadata": { | ||
"tags": [] | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"ccss_stations_snwd_df" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "1d56714b-89c2-4ff7-a0fa-bae9124426e8", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "cb09c66b-b74e-41d4-8c6c-ba6a12d3c4e0", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "Python 3 (ipykernel)", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.11.7" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 5 | ||
} |