Skip to content

Commit

Permalink
Merge pull request #138 from UW-GDA/egagli-patch-50
Browse files Browse the repository at this point in the history
add demo nb to get snotel data from github hosted csvs
  • Loading branch information
egagli authored Feb 21, 2024
2 parents 9a3f7bc + 9ad93c4 commit 380dd7a
Showing 1 changed file with 319 additions and 0 deletions.
319 changes: 319 additions & 0 deletions book/modules/08_Vector_TimeSeries_SNOTEL/08_SNOTEL_download_csvs.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,319 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "ad0998b7-a501-4f9f-a8c4-2ef01a23fb76",
"metadata": {},
"source": [
"# 08 Demo: Using snotel_ccss_stations to get SNOTEL data\n",
"\n",
"UW Geospatial Data Analysis \n",
"CEE467/CEWA567 \n",
"Eric Gagliano "
]
},
{
"cell_type": "markdown",
"id": "5106c82f-297d-46bb-a1c0-0818bfb134e3",
"metadata": {},
"source": [
"**Thanks for checking out this notebook! My hope is that this repository makes it easier to retrieve daily SNOTEL and CCSS data without having to do clunky downloads and conversions. Snow depth / SWE / PRCPSA are in meters, temperatures are in celsius. The only required packages you'll need are geopandas and pandas, the rest of the imports are for applications :)**"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b3959e4b-74c7-426f-b903-449ebfff068e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import pandas as pd\n",
"import geopandas as gpd\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import datetime\n",
"import tqdm\n",
"import contextily as ctx"
]
},
{
"cell_type": "markdown",
"id": "0ec527a0-41b1-449e-8a59-8d456073ceca",
"metadata": {
"tags": []
},
"source": [
"## View all SNOTEL & CCSS stations\n",
"- the [SNOwpack TELemetry (SNOTEL) network](https://www.nrcs.usda.gov/wps/portal/wcc/home/aboutUs/monitoringPrograms/automatedSnowMonitoring/) includes over 800 automated weather stations in the Western U.S. for mountain snowpack observation\n",
"- the [CCSS program](https://water.ca.gov/Programs/Flood-Management/Flood-Data/Snow-Surveys) manages a network of 130 automated snow sensors located in the Szierra Nevada and Shasta-Trinity Mountains"
]
},
{
"cell_type": "markdown",
"id": "dad815f9-c694-4772-b9e8-cdb83f0c9d69",
"metadata": {
"tags": []
},
"source": [
"### Read the geojson stored at https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/all_stations.geojson\n",
"- the daily recurring github action should regularly update the `endDate` column \n",
"- set the index of the geodataframe to the code column\n",
"- let's only look at sites for which we have data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f7c60689-178f-42a2-b122-eacee67f5bcb",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"all_stations_gdf = gpd.read_file('https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/all_stations.geojson').set_index('code')\n",
"all_stations_gdf = all_stations_gdf[all_stations_gdf['csvData']==True]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "adc6b061-823d-44c3-ae81-f6527a23aedd",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"all_stations_gdf"
]
},
{
"cell_type": "markdown",
"id": "43621e35-4326-4cf9-a221-f5f8ea614cbe",
"metadata": {
"tags": []
},
"source": [
"### Use geopandas `GeoDataFrame.explore()` on the `all_stations_gdf` geodataframe to interactively view the stations \n",
"- color by network: red is SNOTEL, blue is CCSS."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1ef6caa1-e841-4e46-8989-afb95af787b5",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"all_stations_gdf.astype(dict(beginDate=str, endDate=str)).explore(column='network',cmap='bwr')"
]
},
{
"cell_type": "markdown",
"id": "aaf197e5-7fd4-4c39-8b67-315de019f343",
"metadata": {
"tags": []
},
"source": [
"## Read a singular CSV: *Paradise, WA*\n",
"- check out information about the [SNOTEL station near Mt. Rainier at Paradise, WA](https://wcc.sc.egov.usda.gov/nwcc/site?sitenum=679)\n",
"- cool plots available at the [Northwest River Forecast Center website](https://www.nwrfc.noaa.gov/snow/snowplot.cgi?AFSW1)"
]
},
{
"cell_type": "markdown",
"id": "1cbbc606-69e5-4898-b9bc-6bfff881215a",
"metadata": {
"tags": []
},
"source": [
"### Place a station code (which you can find in this interactive plot, or by other means) in the url: https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station_id}.csv\n",
"- for SNOTEL stations, this will be of the form {unique number}_{two letter state abbreviation}_SNTL (e.g. 679_WA_SNTL). \n",
"- for CCSS stations, this will be a three letter code (e.g. BLK). \n",
"- use `pd.read_csv()` with `index_col='datetime'` and `parse_dates=True` so we interpret the datetime column as pandas datetime objects"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "82c6bd2e-3d27-4908-ab24-6e61772f4e8a",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"station_id = '679_WA_SNTL'\n",
"paradise_snotel = pd.read_csv(f'https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station_id}.csv',index_col='datetime', parse_dates=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "33bb24d7-44f9-49a5-b728-312495160f50",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"paradise_snotel"
]
},
{
"cell_type": "markdown",
"id": "cc267180-8371-46da-a99d-be42c5837ffc",
"metadata": {
"tags": []
},
"source": [
"### Try a simple plot of snow depth and SWE\n",
"- select the column of interest and use pandas built in `Series.plot()`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cf9f9160-7bbb-4424-b58c-97412e814cb9",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"f,ax=plt.subplots(figsize=(12,5))\n",
"\n",
"paradise_snotel['SNWD'].plot(ax=ax,label='snow depth')\n",
"paradise_snotel['WTEQ'].plot(ax=ax,label='snow water equivalent')\n",
"\n",
"ax.set_xlim(pd.to_datetime(['2017-10-01','2018-09-30']))\n",
"\n",
"ax.grid()\n",
"ax.legend()\n",
"\n",
"ax.set_xlabel('time')\n",
"ax.set_ylabel('snow depth / SWE [meters]')\n",
"ax.set_title('Snow depth and SWE at Paradise, WA \\n(water year 2018)')\n",
"\n",
"f.tight_layout()"
]
},
{
"cell_type": "markdown",
"id": "72fcc194-9e9c-4000-bea1-b3a0a9520604",
"metadata": {
"tags": []
},
"source": [
"## Read a variable from multiple CSVs by looping over a subset of the geodataframe: *CCSS stations*\n",
"- the Sierra Nevada [received a historic amount of snow in 2023](https://www.nps.gov/articles/000/sien-sierranevadamonitor-spring2023.htm)\n",
"- let's explore the magnitude of this season by comparing to the median snow pack"
]
},
{
"cell_type": "markdown",
"id": "9c4696ca-83e1-42e2-994b-679f8d18efcb",
"metadata": {},
"source": [
"### As before, create a list of the stations we are interested in, loop through and add data to a dictionary with the station code as the key, then read into pandas using `pd.DataFrame.from_dict()`\n",
"- create a geodataframe `ccss_stations_gdf` of only CCSS stations from `all_stations_gdf` by creating an index where network equals CCSS\n",
"- loop through the CCSS stations and create a dataframe `ccss_stations_snwd_df`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7d817742-e0f9-43c2-91cd-bbf90ae4ea3b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"ccss_stations_gdf = all_stations_gdf[all_stations_gdf['network']=='CCSS']"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0603ad1a-fd87-4623-ad13-40f9296c1d92",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"ccss_stations_gdf"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d6688f84-bb5e-45a7-8138-771d40c1cfac",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%%time \n",
"station_dict = {}\n",
"\n",
"for station in tqdm.tqdm(ccss_stations_gdf.index):\n",
" try:\n",
" tmp = pd.read_csv(f'https://raw.githubusercontent.com/egagli/snotel_ccss_stations/main/data/{station}.csv',index_col='datetime',parse_dates=True)['SNWD']\n",
" station_dict[station] = tmp\n",
" except:\n",
" print(f'failed to retrieve {station}')\n",
"\n",
"ccss_stations_snwd_df = pd.DataFrame.from_dict(station_dict).dropna(how='all')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0c4a5224-3312-4e8e-8582-484d5049acca",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"ccss_stations_snwd_df"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1d56714b-89c2-4ff7-a0fa-bae9124426e8",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "cb09c66b-b74e-41d4-8c6c-ba6a12d3c4e0",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.7"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

0 comments on commit 380dd7a

Please # to comment.