06-01-OC-marinedata4asia.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![header](https://i.imgur.com/I4ake6d.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# COPERNICUS MARINE OCEANCOLOUR ASIA TRAINING\n",
    "\n",
    "<div style=\"text-align: right\"><i> 06-01-OC-maps+plots-on-ASIA </i></div>\n",
    "\n",
    "***\n",
    "\n",
    "<center><h1>How to visualize maps and time series of chlorophyll surface concentrations </h1></center>\n",
    "\n",
    "***\n",
    "**General Note 1**: Execute each cell through the <button class=\"btn btn-default btn-xs\"><i class=\"icon-play fa fa-play\"></i></button> button from the top MENU (or keyboard shortcut `Shift` + `Enter`).<br>\n",
    "<br>\n",
    "**General Note 2**: If, for any reason, the kernel is not working anymore, in the top MENU, click on the <button class=\"btn btn-default btn-xs\"><i class=\"fa fa-repeat icon-repeat\"></i></button> button. Then, in the top MENU, click on \"Cell\" and select \"Run All Above Selected Cell\".<br>\n",
    "\n",
    "***\n",
    "# Table of contents\n",
    "- [1. Introduction](#1.-Introduction)\n",
    "- [2. About the data](#2.-About-the-data)\n",
    "    - [2.1. Example 1](#2.1.-First-example)\n",
    "        - [2.1.1. Model description](#2.1.1.-Model-description)\n",
    "        - [2.1.2. Get more info on the product](#2.1.2.-Get-more-info-on-the-product)\n",
    "        - [2.1.3. Parameters used for downloading data](#2.1.3.-Parameters-used-for-downloading-data)\n",
    "    - [2.2. Examples 2](#2.2.-Second-example)\n",
    "        - [2.2.1. Model description](#2.2.1.-Model-description)\n",
    "        - [2.2.2. Get more info on the product](#2.2.2.-Get-more-info-on-the-product)\n",
    "        - [2.2.3. Parameters used for downloading data (two datasets)](#2.2.3.-Parameters-used-for-downloading-data-(two-datasets))\n",
    "    - [2.3. Download data with motuclient](#2.3.-Download-data-with-motuclient)\n",
    "- [3. Exercise n.1: Plot of maps](#3.-Exercise-n.1:-Plot-of-maps)\n",
    "    - [3.1. Required modules](#3.1.-Required-modules)\n",
    "    - [3.2. Import the modules](#3.2.-Import-the-modules)\n",
    "    - [3.3. Access the data](#3.3.-Access-the-data)\n",
    "    - [3.4. Set the configuration](#3.4.-Set-the-configuration)\n",
    "    - [3.5. Plot the variables](#3.5.-Plot-the-variables)\n",
    "- [4. Exercise n.2: Plot of time series](#4.-Exercise-n.2:-Plot-of-Time-series)\n",
    "    - [4.1. Required Python version : 3.7](#4.1.-Required-Python-version-:-3.7)\n",
    "    - [4.2. Required modules](#4.2.-Required-modules)\n",
    "    - [4.3. Import the modules](#4.3.-Import-the-modules)\n",
    "    - [4.4. Access the data](#4.4.-Access-the-data)\n",
    "    - [4.5. Set the configuration](#4.5.-Set-the-configuration)\n",
    "    - [4.6. Plot the time series](#4.6.-Plot-the-time-series)\n",
    "    - [4.7. Save the data in csv file format](#4.7.-Save-the-data-in-csv-file-format)\n",
    "- [5. Conclusion](#5.-Conclusion)\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This python notebook contains two examples of visualization of the chlorophyll (Chl) products for the Asian seas.\n",
    "- The first example will show how to make a map of Chlorophyll data using the Near Real Time daily interpolated chlorophyll field.\n",
    "- The second example will show how to plot at two locations in the basin the Reprocessed Chlorophyll daily time series and of the monthly averages."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. About the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1. First example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.1. Model description\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)\n",
    "\n",
    "The first example is based on the product  [**OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_082**](https://resources.marine.copernicus.eu/?option=com_csw&view=details&product_id=OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_082)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the Global Ocean Satellite Observations, ACRI-ST company (Sophia Antipolis, France) is providing Chlorophyll-a and Optics products - present based on the Copernicus-GlobColour processor.\n",
    "Chlorophyll and Bio products refer to Chlorophyll-a, Primary Production (PP) and Phytoplankton Functional types (PFT). Products are based on a multi sensors/algorithms approach to provide to end-users the best estimate. \n",
    "\n",
    "Two **daily Chlorophyll-a products** are distributed:\n",
    "- one limited to the daily observations (called L3),\n",
    "- the other based on a space-time interpolation: the \"Cloud Free\" (called L4).\n",
    "\n",
    "Optics products refer to Reflectance (RRS), Suspended Matter (SPM), Particulate Backscattering (BBP), Secchi Transparency Depth (ZSD), Diffuse Attenuation (KD490) and Absorption Coef. (ADG/CDM).\n",
    "\n",
    "The **spatial resolution** is 4 km. For Chlorophyll, a 1 km over the Atlantic (46°W-13°E , 20°N-66°N) is also available for the Cloud Free product.\n",
    "\n",
    "Products (Daily, Monthly and Climatology) are based on the merging of the sensors SeaWiFS, MODIS, MERIS, VIIRS-SNPP&JPSS1, OLCI-S3A&S3B. For OLCI-S3A, products are also delivered as a single sensor.\n",
    "Recent products are organized in datasets called NRT (Near Real Time) and long time-series in datasets called REP/MY (Multi-Years). The NRT products are provided one day after satellite acquisition and updated a few days after in Delayed Time (DT) to provide a better quality. An uncertainty is given at pixel level for all products.\n",
    "\n",
    "\n",
    "<center><img src=\"./img/OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_082.gif\"></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.2. Get more info on the product\n",
    "\n",
    "1. You can find info on this product and access to the download services in the [dedicated Product Page on Copernicus Marine (CMEMS) Web portal](https://resources.marine.copernicus.eu/?option=com_csw&view=details&product_id=OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_082)\n",
    "<br>\n",
    "2. For **detailed information** about the product, please consult the **document**: [Product User Manual (PUM)](./docs/ProductUserManuals/CMEMS-OC-PUM-009-ALL.pdf)\n",
    "<br>\n",
    "3. For information about **the quality and validation** of the product, please consult the **document**: [Quality Information Document (QUID)](./docs/QualityInformationDocuments/CMEMS-OC-QUID-009-030-032-033-037-081-082-083-085-086-098.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.3. Parameters used for downloading data\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)\n",
    "\n",
    "| Parameter | Value |\n",
    "| :--- | :---|\n",
    "| **Product** | OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_082 |\n",
    "| **Datasets** | dataset-oc-glo-bio-multi-l4-chl_interpolated_4km_daily-rep |\n",
    "| **Frequency** | daily |\n",
    "| **Lat min** | -90 |\n",
    "| **Lat max** | 90 |\n",
    "| **Lon min** | -180 |\n",
    "| **Lon max** | 180 |\n",
    "| **Timesteps** | from  2020-12-25T00:00 to 2020-12-26T00:00 |\n",
    "| **Variables** | CHL | \n",
    "| **Service for downloading** | SUBSETTER |\n",
    "| **Files total dimension** | 299 MB |\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2. Second example\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.1. Model description\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)\n",
    "\n",
    "The second example is based on the product: [**OCEANCOLOUR_GLO_CHL_L4_NRT_OBSERVATIONS_009_033**](https://resources.marine.copernicus.eu/?option=com_csw&view=details&product_id=OCEANCOLOUR_GLO_CHL_L4_NRT_OBSERVATIONS_009_033)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the Global Ocean Satellite Observations', ACRI-ST company (Sophia Antipolis, France) is providing Chlorophyll-a and Optics products - present based on the Copernicus-GlobColour processor. Chlorophyll and Bio products refer to Chlorophyll-a, Primary Production (PP) and Phytoplankton Functional types (PFT). Products are based on a multi sensors/algorithms approach to provide to end-users the best estimate. \n",
    "\n",
    "Two dailies Chlorophyll-a products are distributed: \n",
    "\n",
    "- one limited to the daily observations (called L3), \n",
    "- the other based on a space-time interpolation: the \"Cloud Free\" (called L4). \n",
    "\n",
    "Optics products refer to Reflectance (RRS), Suspended Matter (SPM), Particulate Backscattering (BBP), Secchi Transparency Depth (ZSD), Diffuse Attenuation (KD490) and Absorption Coef. (ADG/CDM). \n",
    "\n",
    "The spatial resolution is 4 km. For Chlorophyll, a 1 km over the Atlantic (46°W-13°E , 20°N-66°N) is also available for the Cloud Free product. Products (Daily, Monthly and Climatology) are based on the merging of the sensors SeaWiFS, MODIS, MERIS, VIIRS-SNPP&JPSS1, OLCI-S3A&S3B. For OLCI-S3A, products are also delivered as a single sensor. Recent products are organized in datasets called NRT (Near Real Time) and long time-series in datasets called REP/MY (Multi-Years). The NRT products are provided one day after satellite acquisition and updated a few days after in Delayed Time (DT) to provide a better quality. An uncertainty is given at pixel level for all products.\n",
    "\n",
    "<center><img src=\"./img/OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_082.gif\"></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.2. Get more info on the product\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)\n",
    "\n",
    "1. You can find info on this product and access to the download services in the [dedicated Product Page on Copernicus Marine (CMEMS) Web portal](https://resources.marine.copernicus.eu/?option=com_csw&view=details&product_id=OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_033)\n",
    "<br>\n",
    "2. For **detailed information** about the product, please consult the **document**: [Product User Manual (PUM)](./docs/ProductUserManuals/CMEMS-OC-PUM-009-ALL.pdf)\n",
    "<br>\n",
    "3. For information about **the quality and validation** of the product, please consult the **document**: [Quality Information Document (QUID)](./docs/QualityInformationDocuments/CMEMS-OC-QUID-009-030-032-033-037-081-082-083-085-086-098.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.3. Parameters used for downloading data (two datasets)\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)\n",
    "\n",
    "| Parameter | Value |\n",
    "| :--- | :---|\n",
    "| **Product** | OCEANCOLOUR_GLO_CHL_L4_NRT_OBSERVATIONS_009_033 |\n",
    "| **Datasets** | dataset-oc-glo-bio-multi-l4-interpolated_4km_daily-rt |\n",
    "| **Frequency** | daily |\n",
    "| **Lat min** | 7 and 35 |\n",
    "| **Lat max** | 10 and 43.5  |\n",
    "| **Lon min** | 102 and 127 |\n",
    "| **Lon max** | 108 and 135 |\n",
    "| **Timesteps** | both from 2020-09-01T00:00 to 2021-05-01T00:00 |\n",
    "| **Variables** | CHL | \n",
    "| **Service for downloading** | SUBSETTER |\n",
    "| **Files total dimension** | 10.3 MB and 38.5 MB |\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3. Download data with motuclient\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "    <b>Get the CMEMS User credentials</b>\n",
    "<hr>\n",
    "Before being able to run the following exercises on your PC and to download the needed data, please be sure to have by your CMEMS User credentials. Otherwise please get them <a href=\"http://marine.copernicus.eu/services-portfolio/register-now/\" target=\"_blank\">here</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download motuclient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data can be alternatively downloaded with **motuclient**, a multiplatform python script to automate extraction of information from the Marine Copernicus portal. To install motuclient:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "!pip install motuclient\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install motuclient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get more info on motuclient visit the related [web page](https://marine.copernicus.eu/faq/what-are-the-motu-and-python-requirements/#motu-client)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### First example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll be able to download data with the following command line that you have to copy to another cell for being executed (**NOTE:** don't forget to set username and password):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "!python -m motuclient --motu http://my.cmems-du.eu/motu-web/Motu \\\n",
    "    --service-id OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_082-TDS \\\n",
    "    --product-id dataset-oc-glo-bio-multi-l4-chl_interpolated_4km_daily-rep \\\n",
    "    --longitude-min -180 --longitude-max 180 \\\n",
    "    --latitude-min -90 --latitude-max 90 \\\n",
    "    --date-min \"2020-12-25 00:00:00\" --date-max \"2020-12-26 00:00:00\" --variable CHL \\\n",
    "    --out-dir ./data --out-name 09-01-A.nc \n",
    "    --user <USERNAME> --pwd <PASSWORD> \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Insert the download command below, if needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install motuclient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "\n",
    "If the directory **data** already exists, comment the following command:\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import os\n",
    "#os.makedirs(\"./data\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -m motuclient --motu http://my.cmems-du.eu/motu-web/Motu \\\n",
    "    --service-id OCEANCOLOUR_GLO_CHL_L4_REP_OBSERVATIONS_009_082-TDS \\\n",
    "    --product-id dataset-oc-glo-bio-multi-l4-chl_interpolated_4km_daily-rep \\\n",
    "    --longitude-min -180 --longitude-max 180 \\\n",
    "    --latitude-min -90 --latitude-max 90 \\\n",
    "    --date-min \"2020-12-25 00:00:00\" --date-max \"2020-12-26 00:00:00\" --variable CHL \\\n",
    "    --out-dir ./data --out-name 09-01-A.nc \\\n",
    "    --user 'user' --pwd 'pwd!' "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check if the file has been correctly downloaded:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "for filename in os.listdir(\"data\"):\n",
    "    print(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Second example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll be able to download data with the following command lines that you have to copy to another cell for being execute (**NOTE:** don't forget to set username and password):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "!python -m motuclient --motu https://nrt.cmems-du.eu/motu-web/Motu \\\n",
    "    --service-id OCEANCOLOUR_GLO_CHL_L4_NRT_OBSERVATIONS_009_033-TDS \\\n",
    "    --product-id dataset-oc-glo-bio-multi-l4-chl_interpolated_4km_daily-rt \\\n",
    "    --longitude-min 102 --longitude-max 108 \\\n",
    "    --latitude-min 7 --latitude-max 10 \\\n",
    "    --date-min \"2020-09-01 00:00:00\" --date-max \"2021-05-01 00:00:00\" \\\n",
    "    --variable CHL \\\n",
    "    --out-dir ./data --out-name \"09-01-B.nc\" \\\n",
    "    --user 'USERNAME' --pwd 'PASSWORD'\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Insert the download command below, if needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -m motuclient --motu https://nrt.cmems-du.eu/motu-web/Motu \\\n",
    "    --service-id OCEANCOLOUR_GLO_CHL_L4_NRT_OBSERVATIONS_009_033-TDS \\\n",
    "    --product-id dataset-oc-glo-bio-multi-l4-chl_interpolated_4km_daily-rt \\\n",
    "    --longitude-min 102 --longitude-max 108 \\\n",
    "    --latitude-min 7 --latitude-max 10 \\\n",
    "    --date-min \"2020-09-01 00:00:00\" --date-max \"2021-05-01 00:00:00\" \\\n",
    "    --variable CHL \\\n",
    "    --out-dir ./data --out-name \"09-01-B.nc\" \\\n",
    "    --user 'user' --pwd 'pwd!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "!python -m motuclient --motu https://nrt.cmems-du.eu/motu-web/Motu \\\n",
    "    --service-id OCEANCOLOUR_GLO_CHL_L4_NRT_OBSERVATIONS_009_033-TDS \\\n",
    "    --product-id dataset-oc-glo-bio-multi-l4-chl_interpolated_4km_daily-rt \\\n",
    "    --longitude-min 127 --longitude-max 135 \\\n",
    "    --latitude-min 35 --latitude-max 43.5 \\\n",
    "    --date-min \"2020-09-01 00:00:00\" --date-max \"2020-12-31 00:00:00\" \\\n",
    "    --variable CHL \\\n",
    "    --out-dir ./data --out-name \"09-01-C.nc\" \\\n",
    "    --user 'USERNAME' --pwd 'PASSWORD'\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Insert the download command below, if needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -m motuclient --motu https://nrt.cmems-du.eu/motu-web/Motu \\\n",
    "    --service-id OCEANCOLOUR_GLO_CHL_L4_NRT_OBSERVATIONS_009_033-TDS \\\n",
    "    --product-id dataset-oc-glo-bio-multi-l4-chl_interpolated_4km_daily-rt \\\n",
    "    --longitude-min 127 --longitude-max 135 \\\n",
    "    --latitude-min 35 --latitude-max 43.5 \\\n",
    "    --date-min \"2020-09-01 00:00:00\" --date-max \"2021-05-01 00:00:00\" \\\n",
    "    --variable CHL \\\n",
    "    --out-dir ./data --out-name \"09-01-C.nc\" \\\n",
    "    --user 'user' --pwd 'pwd!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check if the file has been correctly downloaded:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "for filename in os.listdir(\"data\"):\n",
    "    print(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Exercise n.1: Plot of maps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise we will see how to plot maps of chlorophyll for the Mediterranean Sea\n",
    "\n",
    "### 3.1. Required Python version : 3.7\n",
    "Check your python version with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python --version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If needed, you can install the right Python version in this way:\n",
    "```\n",
    "conda install python=3.7\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2. Required modules\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Module name | Description |\n",
    "| :--- | :---|\n",
    "| **os** | [ Miscellaneous operating system interfaces](https://docs.python.org/3.7/library/os.html) for managing paths, creating directories,... |\n",
    "| **numpy** | [NumPy](https://numpy.org/) is the fundamental package for scientific computing with Python and for managing ND-arrays |\n",
    "| **xarray** | [Xarray](http://xarray.pydata.org/en/stable/) introduces labels in the form of dimensions, coordinates and attributes on top of raw NumPy-like arrays, which allows for a more intuitive, more concise, and less error-prone developer experience. |\n",
    "| **matplotlib** |[Matplotlib](https://matplotlib.org/) is a Python 2D plotting library which produces publication quality figures |\n",
    "| **basemap** |[The matplotlib basemap toolkit](https://matplotlib.org/basemap/) is a library for plotting 2D data on maps in Python. |\n",
    "| **datetime** |[The datetime module](https://docs.python.org/3.8/library/datetime.html) supplies classes for manipulating dates and times in both simple and complex ways. |\n",
    "| **pandas** |[pandas](https://pandas.pydata.org/) is a library providing high-performance, easy-to-use data structures and data analysis tools for Python |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install the needed Python modules (for local installation, can be skipped during Training)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are using the standard *Anaconda* installation, you can install the modules executing the following command in a new cell:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "conda install numpy xarray matplotlib basemap netcdf4 proj4 basemap-data-hires\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2. Import the modules\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before going on, since we are in Jupyter Notebook, we need to set the environment in order to properly load some of the required libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To avoid the warning messages during the execution and installation process, at first remove them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's time to load all the other required libraries. In case you get an error during this step, try setting the variable `os.environ['PROJ_LIB]` to the shared path of your conda environment. For example by executing in a python cell (don't forget to replace `<PATH_TO_ANACONDA>` and  `<YOUR_ENV>` with the proper values):\n",
    "\n",
    "```\n",
    "os.environ['PROJ_LIB'] = \"<PATH_TO_ANACONDA>/envs/<YOUR_ENV>/share/proj\"\n",
    "\n",
    "os.environ['PROJ_LIB'] = \"/opt/conda/share/proj\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import xarray as xr\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.basemap import Basemap\n",
    "import datetime\n",
    "import pandas\n",
    "from matplotlib.colors import LogNorm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Don't change the following constants, which define the training and the notebook codes*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "REGION_CODE = \"09\"\n",
    "NB_CODE = \"01\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**checkDir**: function for creating a path, if needed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def checkDir(outPath):\n",
    "    if not os.path.exists(outPath):\n",
    "        os.makedirs(outPath)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**getRangeIndexes**: function for getting the indexes of the array *arr* between the *var_min* and *var_max* values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def getRangeIndexes(arr, var_min, var_max):\n",
    "    return np.where((arr >= var_min) & (arr <= var_max))[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the paths:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Path for netcdf files\n",
    "data_path = './data'\n",
    "# Path for the output files (images, etc)\n",
    "out_path = './out/'+REGION_CODE+'-'+NB_CODE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the current directories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for filename in os.listdir():\n",
    "    print(filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create directories\n",
    "checkDir(data_path)\n",
    "checkDir(out_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check if the new directories have been created and if the data files are available:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for path in [data_path, out_path]:\n",
    "    if os.path.isdir(path):\n",
    "        print(path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for filename in os.listdir(data_path):\n",
    "    print(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3. Access the data\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Input netcdf file\n",
    "oc_f = \"09-01-A.nc\"\n",
    " \n",
    "# Build the complete nc path\n",
    "oc_nc = os.path.join(data_path, oc_f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Open the nc dataset\n",
    "oc_ds = xr.open_dataset(oc_nc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get info about the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "oc_ds.info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And about the variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "oc_ds.data_vars"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... the coordinates:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "oc_ds.coords"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4. Set the configuration\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the coordinate names (used later for accessing the data)\n",
    "lon_name = \"lon\"\n",
    "lat_name = \"lat\"\n",
    "time_name = \"time\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the variable names\n",
    "var_name = \"CHL\" # chlorophyll"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.5. Plot the variables\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure the variables for the plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data to plot\n",
    "var_sel = oc_ds[var_name]\n",
    "\n",
    "# --- Set up the arrays of coordinates for the selected dataset ---\n",
    "# \n",
    "lats = var_sel[lat_name]\n",
    "lons = var_sel[lon_name]\n",
    "times = var_sel[time_name]\n",
    "    \n",
    "# --- Set the area: choose the min and max values for the coordinates ---\n",
    "# (and comment with the symbol # the unnecessary ones!):                                                               \n",
    "                                                                 \n",
    "\n",
    "# Set lat-lon limits - FULL AREA\n",
    "lat_min = lats.min()\n",
    "lat_max = lats.max()\n",
    "lon_min = lons.min()\n",
    "lon_max = lons.max()\n",
    "\n",
    "## Set lat-lon limits - CUSTOM:  focus on Asian Waters\n",
    "lon_min = 35\n",
    "lon_max = 170\n",
    "lat_min = -20\n",
    "lat_max = 60\n",
    "\n",
    "\n",
    "# set start and end date\n",
    "start_date = times.min()\n",
    "end_date = times.max()\n",
    "\n",
    "# Set the variable min and max values for the plot and the colorbar (otherwise assign None):\n",
    "# min_value, max_value = None, None\n",
    "min_value, max_value = 0, 6.5\n",
    "\n",
    "# -- Set the time period --- \n",
    "plot_date = np.datetime64(datetime.datetime(2020,12,25,0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot configuration\n",
    "width_inch = 16\n",
    "height_inch = 8\n",
    "\n",
    "# Map configuration\n",
    "map_config = { \n",
    "#     \"projection\": \"merc\",\n",
    "    \"llcrnrlat\": lat_min,\n",
    "    \"llcrnrlon\": lon_min,\n",
    "    \"urcrnrlat\": lat_max,\n",
    "    \"urcrnrlon\": lon_max,\n",
    "    \"resolution\": 'i',\n",
    "    \"epsg\": 4326\n",
    "}\n",
    "\n",
    "# Axes labels\n",
    "fontsize = 14\n",
    "xlabel = 'longitude [deg]'\n",
    "ylabel = 'latitude [deg]'\n",
    "xlabelpad = 30\n",
    "ylabelpad = 60\n",
    "\n",
    "# Colorbar configuration\n",
    "cmap = \"Spectral_r\"\n",
    "cbar_position = \"right\"\n",
    "\n",
    "title_fontstyle = {\n",
    "    \"fontsize\": \"14\",\n",
    "    \"fontstyle\": \"italic\",\n",
    "    \"fontweight\": \"bold\",\n",
    "    \"pad\": 30\n",
    "}\n",
    "\n",
    "label_fontstyle = {\n",
    "    \"fontsize\": \"12\",\n",
    "    \"labelpad\": 30\n",
    "}\n",
    "\n",
    "# Set the coordinates indexes\n",
    "lat_indexes = getRangeIndexes(lats, lat_min, lat_max)\n",
    "lon_indexes = getRangeIndexes(lons, lon_min, lon_max)\n",
    "time_indexes = getRangeIndexes(times, start_date, end_date)\n",
    "\n",
    "# Get the selected coordinates:\n",
    "lats_sel = lats[lat_indexes]\n",
    "lons_sel = lons[lon_indexes]\n",
    "\n",
    "# Get the timestep\n",
    "timestep = np.datetime_as_string(plot_date,'h')\n",
    "\n",
    "# Get the data\n",
    "data = var_sel[time_indexes[0], lat_indexes, lon_indexes]\n",
    "# Flip north-south (why is it needed?)\n",
    "data = np.flipud(data)\n",
    "\n",
    "# Initialise the figure\n",
    "plt.figure(figsize=(width_inch, height_inch))\n",
    "map = Basemap(**map_config)\n",
    "\n",
    "## imshow the data\n",
    "min_value = data.min() if min_value is None else min_value\n",
    "max_value = data.max() if max_value is None else max_value\n",
    "\n",
    "# # imshow with linear range\n",
    "# imshow= map.imshow(data, vmin=min_value, vmax=max_value, cmap=cmap)\n",
    "\n",
    "# imshow with log range\n",
    "# min fixed \n",
    "min_value=0.01\n",
    "imshow= map.imshow(data, vmin=min_value, vmax=max_value, cmap=cmap, norm=LogNorm())\n",
    "\n",
    "## draw meridians and parallels\n",
    "step_lat = float((lat_max - lat_min) / 5)\n",
    "step_lon = float((lon_max - lon_min) / 5)\n",
    "parallels = np.arange(lat_min, lat_max, step_lat)\n",
    "meridians = np.arange(lon_min, lon_max, step_lon) \n",
    "map.drawmeridians(meridians, labels=[0,0,0,1], fmt=\"%2.1f\")\n",
    "map.drawparallels(parallels, labels=[1,0,0,0], fmt=\"%2.1f\")\n",
    "\n",
    "## draw colorbar\n",
    "map.colorbar(imshow, cbar_position)\n",
    "\n",
    "## draw countries...\n",
    "map.drawcountries(linewidth=0.5, color='slategray')\n",
    "map.fillcontinents(color='lightgray',lake_color='aqua')\n",
    "\n",
    "# set title\n",
    "title_sel = var_sel.long_name\n",
    "var_str = \"{} [{}]\".format(title_sel, var_sel.units)\n",
    "title = ' - '.join((var_str, timestep))\n",
    "    \n",
    "## draw title and axes labels\n",
    "plt.title(title, **title_fontstyle)\n",
    "plt.xlabel(xlabel, labelpad=xlabelpad, fontsize=12)\n",
    "plt.ylabel(ylabel, labelpad=ylabelpad, fontsize=12)    \n",
    "    \n",
    "# output file\n",
    "output_file = os.path.join(out_path,title.translate(str.maketrans(' ,','__'))) + \".png\"\n",
    "\n",
    "\n",
    "# save the output file\n",
    "plt.savefig(output_file)\n",
    "plt.show()\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you can't see easily the maps, you can disable the scrolling. \n",
    "Choose the previous cell (the one with the maps), then select in the menu: **Cell &rarr; Current Output &rarr; Toggle Scrolling**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "\n",
    "### LET'S TRY: Change the plot!\n",
    "\n",
    "If you want, you can change the above plot.\n",
    "\n",
    "For example, you could:\n",
    "- change the area\n",
    "- change the min/max values for the chosen variable\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Exercise n.2: Plot of Time series\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise we will see how to plot the time series of chlorophyll at two locations of the Asian waters.\n",
    "The first point is located in the **Sea of Japan**,  the second in the **South China Sea**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1. Required Python version : 3.7\n",
    "Check your python version with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python --version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If needed, you can install the right Python version in this way:\n",
    "```\n",
    "conda install python=3.7\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2. Required modules\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The list of modules required to run this exercise is reported in the table below:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Module name | Description |\n",
    "| :--- | :---|\n",
    "| **os** | [ Miscellaneous operating system interfaces](https://docs.python.org/3.7/library/os.html) for managing paths, creating directories,... |\n",
    "| **numpy** | [NumPy](https://numpy.org/) is the fundamental package for scientific computing with Python and for managing ND-arrays |\n",
    "| **xarray** | [Xarray](http://xarray.pydata.org/en/stable/) introduces labels in the form of dimensions, coordinates and attributes on top of raw NumPy-like arrays, which allows for a more intuitive, more concise, and less error-prone developer experience. |\n",
    "| **matplotlib** |[Matplotlib](https://matplotlib.org/) is a Python 2D plotting library which produces publication quality figures |\n",
    "| **basemap** |[The matplotlib basemap toolkit](https://matplotlib.org/basemap/) is a library for plotting 2D data on maps in Python. |\n",
    "| **datetime** |[The datetime module](https://docs.python.org/3.8/library/datetime.html) supplies classes for manipulating dates and times in both simple and complex ways. |\n",
    "| **pandas** |[pandas](https://pandas.pydata.org/) is a library providing high-performance, easy-to-use data structures and data analysis tools for Python |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Install the needed Python modules (for local installation, can be skipped during Training)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are using the standard *Anaconda* installation, you can install the modules executing the following command in a new cell:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "conda install numpy xarray matplotlib basemap netcdf4 basemap-data-hires\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3. Import the modules\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before going on, since we are in Jupyter Notebook, we need to set the environment in order to properly load some of the required libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To avoid the warning messages during the execution and installation process, at first remove them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's time to load all the other required libraries. In case you get an error during this step, try setting the variable `os.environ['PROJ_LIB]` to the shared path of your conda environment. For example by executing in a python cell (don't forget to replace `<PATH_TO_ANACONDA>` and  `<YOUR_ENV>` with the proper values):\n",
    "\n",
    "```\n",
    "os.environ['PROJ_LIB'] = \"<PATH_TO_ANACONDA>/envs/<YOUR_ENV>/share/proj\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import xarray as xr\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.basemap import Basemap\n",
    "import datetime\n",
    "import pandas as pd\n",
    "import matplotlib.dates as mdates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Don't change the following constants, which define the training and the notebook codes*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "REGION_CODE = \"09\"\n",
    "NB_CODE = \"01\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**checkDir**: function for creating a path, if needed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def checkDir(outPath):\n",
    "    if not os.path.exists(outPath):\n",
    "        os.makedirs(outPath)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**getRangeIndexes**: function for getting the indexes of the array *arr* between the *var_min* and *var_max* values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def getRangeIndexes(arr, var_min, var_max):\n",
    "    return np.where((arr >= var_min) & (arr <= var_max))[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the paths:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Path for netcdf files\n",
    "data_path = './data'\n",
    "\n",
    "# Path for the output files (images, etc)\n",
    "out_path = './out/'+REGION_CODE+'-'+NB_CODE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.4. Access the data\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Input netcdf file\n",
    "chl_f1 = \"09-01-B.nc\"\n",
    "chl_f2 = \"09-01-C.nc\"\n",
    " \n",
    "# Build the complete nc path\n",
    "chl_nc1 = os.path.join(data_path, chl_f1)\n",
    "chl_nc2 = os.path.join(data_path, chl_f2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Open the nc datasets\n",
    "chl_ds1 = xr.open_dataset(chl_nc1)\n",
    "chl_ds2 = xr.open_dataset(chl_nc2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get info about the datasets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chl_ds1.info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chl_ds2.info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.5. Set the configuration\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the variable names (used later for accessing the data)\n",
    "lon_name = \"lon\"\n",
    "lat_name= \"lat\"\n",
    "time_name = \"time\"\n",
    "chl_name = \"CHL\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.6. Plot the time series\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Choose the points of interest:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the points of interest x is lon y is lat\n",
    "point1 = {'x': 106, 'y': 8.43}\n",
    "point2 = {'x': 133, 'y': 38.34}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Extract the dataset for the selected points:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract the sub-dataset for the first defined point\n",
    "ds_sel1 = chl_ds1.sel(lat=point1['y'], lon=point1['x'], method='nearest')\n",
    "ds_sel1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract the sub-dataset for the second defined point\n",
    "ds_sel2 = chl_ds2.sel(lat=point2['y'], lon=point2['x'], method='nearest')\n",
    "ds_sel2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Extract the chlorophyll for the selected time series:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Extract the chlorophyll for the first point\n",
    "chl_sel1 = ds_sel1[chl_name].round(3)\n",
    "chl_sel1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Extract the chlorophyll for the second point\n",
    "chl_sel2 = ds_sel2[chl_name].round(3)\n",
    "chl_sel2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Convert the two datasets into a pandas DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# define two labels\n",
    "fname1 = \"CHL (%s, %s)\" % (point1[\"x\"], point1[\"y\"])\n",
    "fname2 = \"CHL (%s, %s)\" % (point2[\"x\"], point2[\"y\"])\n",
    "\n",
    "# create dates np.array for the x axis\n",
    "dates=[]\n",
    "for i in range(len(chl_sel1.time.values)) : \n",
    "    dates.append(np.datetime64(chl_sel1.time.values[i]))\n",
    "dates= np.array(dates)   \n",
    "\n",
    "# create a dataframe with data of the first and second points\n",
    "df = pd.DataFrame({\"Dates\": dates, \n",
    "                    fname1: chl_sel1,\n",
    "                    fname2: chl_sel2})\n",
    "# and let’s set the Dates as the DataFrame’s index.\n",
    "df = df.set_index('Dates')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Have a look at the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## print just a section of this dataframe\n",
    "df[100:200:1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print some summary statitics \n",
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Compute monthly averages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# to compute monthly averages we will use the DataFrame’s resample() method\n",
    "# this splits the DatetimeIndex into time bins and groups the data by time bin\n",
    "# Resample to monthly frequency, aggregating with mean\n",
    "df_mean = df.resample(\"M\").mean()\n",
    "\n",
    "# print some summary statitics \n",
    "df_mean.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we will use the location in the panda data frame to define the x-range\n",
    "\n",
    "# Start and end of the date range to plot\n",
    "start, end = '2020', '2021'\n",
    "\n",
    "#start, end = '2014-01', '2018-06'\n",
    "\n",
    "# prepare the plot \n",
    "fig, ax = plt.subplots(figsize=(20,12))\n",
    "\n",
    "\n",
    "\n",
    "# plot the first column of the pandas dataframe for the daility times series \n",
    "ax.plot(df.loc[start:end, fname1],\n",
    "        linestyle=':', lw=1, \n",
    "        marker='o' ,color=\"tomato\",label=fname1+'-Daily')\n",
    "\n",
    "# add to the plot the second column of the pandas dataframe\n",
    "ax.plot(df.loc[start:end, fname2],\n",
    "        linestyle=':', lw=1, \n",
    "        marker='o',  color=\"royalblue\",label=fname2+'-Daily')\n",
    "\n",
    " \n",
    "# now plot also the  monthly averages\n",
    "# comment with # the \"\"\" at the start and botton of this section to uncomment it\n",
    "#\"\"\"\n",
    "ax.plot(df_mean.loc[start:end, fname1],\n",
    "        linestyle='-', lw=3, color=\"darkred\",label=fname1+'-Monthly')\n",
    "ax.plot(df_mean.loc[start:end, fname2],\n",
    "        linestyle='-', lw=3,color=\"darkblue\",label=fname2+'-Monthly')\n",
    "#\"\"\"\n",
    "\n",
    "# format the x axis ticks\n",
    "years = mdates.YearLocator()   # every year\n",
    "months = mdates.MonthLocator()  # every month\n",
    "years_fmt = mdates.DateFormatter('%Y')\n",
    "ax.xaxis.set_major_locator(years)\n",
    "ax.xaxis.set_major_formatter(years_fmt)\n",
    "ax.xaxis.set_minor_locator(months)\n",
    "#plot.set_xlim(datemin, datemax)\n",
    "ax.format_xdata = mdates.DateFormatter('%Y-%m-%d')\n",
    "\n",
    "# format the y axis and ticks \n",
    "ax.set_ylabel('CHL')\n",
    "# set the Y on LOG: COMMENT FOR LINEAR RANGE\n",
    "ax.set_yscale(\"log\")\n",
    "ax.set_yticks([0.1,1,10])\n",
    "ax.set_yticklabels([\"0.1\",\"1\",\"10\"])\n",
    "\n",
    "# set the grid\n",
    "ax.grid(True)\n",
    "\n",
    "# set the legend\n",
    "ax.legend();\n",
    "\n",
    "# set title\n",
    "title = ds_sel1[\"CHL\"].long_name + \" in the Sea of Japan and South China Sea\"\n",
    "ax.set_title(title);\n",
    "\n",
    "# output file\n",
    "output_file = os.path.join(out_path,ds_sel1[\"CHL\"].long_name.replace(' ','_')) + \".png\"\n",
    "\n",
    "# save the output file\n",
    "plt.savefig(output_file)\n",
    "\n",
    "# plot!\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "\n",
    "### LET'S TRY: Change the plot!\n",
    "\n",
    "If you want, you can change the above plot.\n",
    "\n",
    "For example, you could:\n",
    "- plot the monthly means    \n",
    "- change the Start and end of the date range to plot a portion of the time series    \n",
    "- change the Y axis from LOG to LINEAR\n",
    "    \n",
    "Go back to configure the variables for the plots and then re-execute the cell for generating the plot. Try!\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.7. Save the data in csv file format\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save the dataframe to a csv file\n",
    "fname_table = \"chl_data.csv\"\n",
    "output_table =  os.path.join(out_path, fname_table)\n",
    "df.to_csv(output_table)\n",
    "\n",
    "# ...then verify if the file exists\n",
    "if os.path.isfile(output_table):\n",
    "    print(output_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Go back to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "    <b>CONGRATULATIONS</b><br>\n",
    "  \n",
    "--- \n",
    "\n",
    "#### And thank you for your attention! :) We hope you enjoyed this training on the Asian Seas Ocean Colour satellite data provided by Copernicus Marine Service, for free, thanks to the European Commission.\n",
    "\n",
    "#### Now let's try to download new data and variables and to access and visualize them... you can try to make new maps and plots... and don't forget to try to the other [Copernicus Marine products](http://marine.copernicus.eu/services-portfolio/access-to-products/?option=com_csw&task=results)\n",
    "\n",
    "This training course is over but we'd love to hear from you about how we could improve it (topics, tools, storytelling, format, speed etc). \n",
    "\n",
    "We do thank you in advance for your kind collaboration :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "finalized": {
   "timestamp": 1575910793989,
   "trusted": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}