Skip to content

Commit

Permalink
DOC: new user guide structure
Browse files Browse the repository at this point in the history
  • Loading branch information
nghi-truyen committed Jan 17, 2025
1 parent 51ce87e commit a90791b
Showing 29 changed files with 575 additions and 309 deletions.
3 changes: 0 additions & 3 deletions doc/source/release/1.0.0-notes.rst
Original file line number Diff line number Diff line change
@@ -673,9 +673,6 @@ New options have been added to the setup:
This options (one per atmospheric data type) can be used to specify how are stored the atmospheric
data files (i.e., the directories architecture).

.. hint::
See the :ref:`user_guide.others.input_data_convention` convention.

It can be specified as follows:

.. code-block:: yaml
Original file line number Diff line number Diff line change
@@ -1,125 +1,16 @@
.. _user_guide.quickstart.cance_first_simulation:
.. _user_guide.in_depth.classical_calibration_io:

========================
Cance - First Simulation
========================
========================================
Classical Calibration and I/O Operations
========================================

This first tutorial with `smash` will be carried out on a French catchment, **the Cance at Sarras**, a right bank tributary
of the Rhône river. This catchment was chosen for this first tutorial because its moderate size (380 km²)
enables fast computations at a spatial scale of 1km², and because it is well modeled with a low complexity
approach. This tutorial aims to

- provide an overview of the input data required for modelling with `smash`,

- explain how to perform in Python a simulation and a simple model optimization from discharge data at a given gauging station.

.. image:: ../../_static/cance.png
:width: 600
:align: center

Required data
-------------

Before you can start using `smash`, you need to download all the data required to run a simulation on this catchment.

.. button-link:: https://smash.recover.inrae.fr/dataset/Cance-dataset.tar
:color: primary
:shadow:
:align: center

**Download**

If the download was successful, a file named ``Cance-dataset.tar`` should be available. We can switch to the directory where this file has been
downloaded and extract it using the following command:

.. code-block:: shell
tar xf Cance-dataset.tar
Now a folder called ``Cance-dataset`` should be accessible and contain the following files and folders for various spatio-temporal data:

- ``France_flwdir.tif``
A GeoTiff file containing the flow direction data,
- ``gauge_attributes.csv``
A csv file containing the gauge attributes (gauge coordinates, drainage area and code),
- ``prcp``
A directory containing precipitation data in GeoTiff format with the following directory structure: ``%Y/%m/%d``
(``2014/09/15``),
- ``pet``
A directory containing daily interannual potential evapotranspiration data in GeoTiff format,
- ``qobs``
A directory containing the observed discharge data in csv format.

Flow direction
**************

The flow direction file is a mandatory input in order to create a mesh, its associated drainage plan :math:`\mathcal{D}_{\Omega}(x)`, and the localization on the mesh of the gauging stations that we want to model. Here,
the ``France_flwdir.tif`` contains the flow direction data on the whole France, at a spatial resolution of 1km² using a Lambert-93 projection
(**EPSG: 2154**). `smash` is using the following D8 convention for the flow direction.
.. warning::
This section is in development.

.. image:: ../../_static/flwdir_convention.png
:align: center
:width: 175

.. note::

The flow directions should not contain sink(s), i.e. consecutive cells flowing toward each other.
It is therefore important to ensure that flow directions are consistent from upstream to downstream.

Gauge attributes
****************
.. TODO: add i/o guide; simplify this tutorial, moving some parts (e.g., setup, mesh creation) into quickstart.
To create a mesh containing information from the stations in addition to the flow direction file, gauge attributes are mandatory. The gauge
attributes correspond to the spatial coordinates, the drainage area and the code of each gauge. The spatial coordinates must be in the same unit
and projection as the flow direction file (**meter** and **Lambert 93** respectively in our case), the drainage area in **square meter** (or **square kilometer** but it will need
to be converted later). The gauge code can be any code that can be used to identify the station. The ``gauge_attributes.csv`` file has been
filled in to provide this information for the 3 gauging stations of the Cance catchment.

.. note::

We don't use the csv file directly in `smash`, we only use the data it contains. So it's possible to store this data in another format as long
as it can be read with Python.

Precipitation
*************

Precipitation data is mandatory. `smash` expects a precipitation file per time step whose name contains a date in the following format
``%Y%m%d%H%M``. The file must be in GeoTiff format at a resolution and projection identical to the flow direction file. Any unit can be chosen
as long as it can be converted into a millimetre using a simple conversion factor (the unit used in this dataset is tenth of a millimetre).
Regarding the structure of the precipitation folder, there is no strict rule, by default `smash` will fetch all the ``tif`` files in a folder
provided by the user (i.e. ``prcp``). However, when simulating a large number of time steps, we recommend sorting the files as much as possible to
speed up access when reading those (ex. ``%Y/%m/%d``, ``2014/09/15``).

.. note::

As you may have seen when opening any precipitation file, the data has already been cropped over the catchment area. This has been done
simply to reduce the size of the files. It is possible to work with files whose spatial extent is different from the catchment area.
`smash` will automatically crop to the correct area when the file is read.

Potential evapotranspiration
****************************

Potential evapotranspiration data is mandatory. The way in which potential evapotranspiration data is processed is identical to the
precipitation. One difference to note is that instead of working with one potential evapotranspiration file per time step, it is possible to
work with daily interannual data, which therefore requires a file per day whose name contains a date in the following format ``%m%d``.
Here, we provided daily interannual potential evapotranspiration data.

Observed discharge
******************

Observed discharge is optional in case of simulation but mandatory in case of model calibration. `smash` expects a single-column csv file for each gauge
whose name contains the gauge code provided in the ``gauge_attributes.csv`` file. The header of the column is the first time step of the time series,
the data is the observed discharge in **cubic meter per second** and any negative value in the series will be interpreted as no-data.

.. note::

It is not necessary to restrict the observed discharge series to the simulation period. It is possible to provide a time series covering a larger time window over which `smash`
will only read the lines corresponding to dates after the starting date provided in the header.

Now that a brief tour of the necessary data has been done, we can open a Python interface. The current working directory
will be assumed to be the directory where the ``Cance-dataset`` is located.

Open a Python interface:
This tutorial uses the :ref:`Cance dataset <user_guide.demo_data.cance>` to explain how to perform a simulation and a simple model optimization from discharge data at a given gauging station.
We begin by opening a Python interface:

.. code-block:: shell
@@ -326,7 +217,7 @@ To get into more details, this ``mesh`` is composed of:
plt.imshow(mesh["flwdir"]);
plt.colorbar(label="Flow direction (D8)");
@savefig user_guide.quickstart.cance_first_simulation.flwdir.png
@savefig user_guide.in_depth.classical_calibration_io.flwdir.png
plt.title("Cance - Flow direction");
.. hint::
@@ -340,7 +231,7 @@ To get into more details, this ``mesh`` is composed of:
plt.imshow(mesh["flwdst"]);
plt.colorbar(label="Flow distance (m)");
@savefig user_guide.quickstart.cance_first_simulation.flwdst.png
@savefig user_guide.in_depth.classical_calibration_io.flwdst.png
plt.title("Cance - Flow distance");
- ``flwacc``
@@ -350,7 +241,7 @@ To get into more details, this ``mesh`` is composed of:
plt.imshow(mesh["flwacc"]);
plt.colorbar(label="Flow accumulation (m²)");
@savefig user_guide.quickstart.cance_first_simulation.flwacc.png
@savefig user_guide.in_depth.classical_calibration_io.flwacc.png
plt.title("Cance - Flow accumulation");
- ``npar``, ``ncpar``, ``cscpar``, ``cpar_to_rowcol``, ``flwpar``
@@ -362,7 +253,7 @@ To get into more details, this ``mesh`` is composed of:
mesh["npar"], mesh["ncpar"], mesh["cscpar"], mesh["cpar_to_rowcol"]
plt.imshow(mesh["flwpar"]);
plt.colorbar(label="Flow partition (-)");
@savefig user_guide.quickstart.cance_first_simulation.flwpar.png
@savefig user_guide.in_depth.classical_calibration_io.flwpar.png
plt.title("Cance - Flow partition");
- ``nac``, ``active_cell``
@@ -374,7 +265,7 @@ To get into more details, this ``mesh`` is composed of:
mesh["nac"]
plt.imshow(mesh["active_cell"]);
plt.colorbar(label="Active cell (-)");
@savefig user_guide.quickstart.cance_first_simulation.active_cell.png
@savefig user_guide.in_depth.classical_calibration_io.active_cell.png
plt.title("Cance - Active cell");
- ``ng``, ``gauge_pos``, ``code``, ``area``, ``area_dln``
@@ -394,7 +285,7 @@ An important step after generating the ``mesh`` is to check that the stations ha
for pos in mesh["gauge_pos"]:
base[pos[0], pos[1]] = 1
plt.imshow(base, cmap="Set1_r");
@savefig user_guide.quickstart.cance_first_simulation.gauge_position.png
@savefig user_guide.in_depth.classical_calibration_io.gauge_position.png
plt.title("Cance - Gauge position");
.. ipython:: python
@@ -459,7 +350,7 @@ Mesh
model.mesh.nrow, model.mesh.ncol, model.mesh.nac
plt.imshow(model.mesh.flwdir);
plt.colorbar(label="Flow direction (D8)");
@savefig user_guide.quickstart.cance_first_simulation.model_flwdir.png
@savefig user_guide.in_depth.classical_calibration_io.model_flwdir.png
plt.title("Cance - Flow direction");
.. note::
@@ -478,7 +369,7 @@ precipitation for an arbitrary time step.
plt.imshow(model.atmos_data.prcp[..., 1200]);
plt.colorbar(label="Precipitation ($mm/h$)");
@savefig user_guide.quickstart.cance_first_simulation.prcp.png
@savefig user_guide.in_depth.classical_calibration_io.prcp.png
plt.title("Precipitation");
Or masked on the active cells of the catchment
@@ -492,7 +383,7 @@ Or masked on the active cells of the catchment
)
plt.imshow(ma_prcp);
plt.colorbar(label="Precipitation ($mm/h$)");
@savefig user_guide.quickstart.cance_first_simulation.ma_prcp.png
@savefig user_guide.in_depth.classical_calibration_io.ma_prcp.png
plt.title("Masked precipitation");
The spatial average of precipitation (``mean_prcp``) and potential evapotranspiration (``mean_pet``) over each gauge are also computed
@@ -506,7 +397,7 @@ and stored in `Model.atmos_data <smash.Model.atmos_data>`. They are `numpy.ndarr
plt.grid(ls="--", alpha=.7);
plt.legend();
plt.xlabel("Time step");
@savefig user_guide.quickstart.cance_first_simulation.mean_prcp_pet.png
@savefig user_guide.in_depth.classical_calibration_io.mean_prcp_pet.png
plt.title(
f"Mean precipitation and potential evapotranspiration at gauge {code}"
);
@@ -524,7 +415,7 @@ the observed discharge (``q``). The observed discharge is a `numpy.ndarray` of s
plt.grid(ls="--", alpha=.7);
plt.xlabel("Time step");
plt.ylabel("Discharge ($m^3/s$)")
@savefig user_guide.quickstart.cance_first_simulation.qobs.png
@savefig user_guide.in_depth.classical_calibration_io.qobs.png
plt.title(
f"Observed discharge at gauge {code}"
);
@@ -638,7 +529,7 @@ Once the forward run has been completed, we can visualize the simulated discharg
plt.ylabel("Discharge ($m^3/s$)");
plt.grid(ls="--", alpha=.7);
plt.legend();
@savefig user_guide.quickstart.cance_first_simulation.forward_run_q.png
@savefig user_guide.in_depth.classical_calibration_io.forward_run_q.png
plt.title(f"Observed and simulated discharge at gauge {code}");
As the hydrograph shows, the simulated discharge is quite different from the observed discharge at this gauge. Obviously, we ran a forward run with the default `smash` rainfall-runoff
@@ -672,7 +563,7 @@ And visualize again the simulated discharge compared to the observed discharge,
plt.ylabel("Discharge ($m^3/s$)");
plt.grid(ls="--", alpha=.7);
plt.legend();
@savefig user_guide.quickstart.cance_first_simulation.optimize_q.png
@savefig user_guide.in_depth.classical_calibration_io.optimize_q.png
plt.title(f"Observed and simulated discharge at gauge {code}");
Of course, the hydrological model optimization problem is a complex one and there are many strategies that can be employed depending on the modeling goals and data available. Here, for a first tutorial, we have run a simple optimization with the function's
@@ -722,8 +613,6 @@ and read back using the `smash.io.save_model` and `smash.io.read_model` function
model = smash.io.read_model("model.hdf5")
model
This concludes this first tutorial on `smash`. The next quickstart tutorial will cover all of mainland France.

.. ipython:: python
:suppress:
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.. _user_guide.classical_uses.hydrograph_segmentation:

=======================
Hydrograph Segmentation
=======================

.. warning::
This section is in development.
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.. _user_guide.classical_uses.hydrological_signatures:

=======================
Hydrological Signatures
=======================

.. warning::
This section is in development.
Original file line number Diff line number Diff line change
@@ -1,51 +1,12 @@
.. _user_guide.quickstart.france_large_domain_simulation:
.. _user_guide.quickstart.large_domain_simulation:

================================
France - Large Domain Simulation
================================
=======================
Large Domain Simulation
=======================

This second tutorial on `smash` aims to perform a simulation over the whole of metropolitan France with a simple model structure. The objective, compared to the first tutorial, is to create a mesh over a large spatial domain, to perform a forward run and to visualize the simulated discharge over the entire domain.

.. image:: ../../_static/france.png
:width: 350
:align: center

Required data
-------------

You need first to download all the required data.

.. button-link:: https://smash.recover.inrae.fr/dataset/France-dataset.tar
:color: primary
:shadow:
:align: center

**Download**

If the download was successful, a file named ``France-dataset.tar`` should be available. We can switch to the directory where this file has been
downloaded and extract it using the following command:

.. code-block:: shell
tar xf France-dataset.tar
Now a folder called ``France-dataset`` should be accessible and contain the following files and folders:

- ``France_flwdir.tif``
A GeoTiff file containing the flow direction data,
- ``prcp``
A directory containing precipitation data in GeoTiff format with the following directory structure: ``%Y/%m/%d``
(``2012/01/01``),
- ``pet``
A directory containing daily interannual potential evapotranspiration data in GeoTiff format,

In this dataset, there are no gauge attributes or observed discharge, we are only interested in performing a forward run on a domain without
optimization beforehand.

We can open a Python interface. The current working directory will be assumed to be the directory where
the ``France-dataset`` is located.

Open a Python interface:
This tutorial aims to perform a simulation over the whole of metropolitan France with a simple model structure using the :ref:`France dataset <user_guide.demo_data.france>`.
The objective is to create a mesh over a large spatial domain, to perform a forward run and to visualize the simulated discharge over the entire domain.
We begin by opening a Python interface:

.. code-block:: shell
@@ -75,8 +36,8 @@ Model creation
Model setup creation
********************

The ``setup`` dictionary is pretty similar to the one used for the **Cance** tutorial except that we do not read observed discharge and the
simulation period is different.
The ``setup`` dictionary is pretty similar to the one used for the :ref:`Cance <user_guide.demo_data.cance>` tutorial
except that we do not read observed discharge and the simulation period is different.

.. ipython:: python
@@ -125,14 +86,14 @@ We can visualize the shape of the ``mesh``, the flow direction and the flow accu
plt.imshow(mesh["flwdir"]);
plt.colorbar(label="Flow direction (D8)");
@savefig user_guide.quickstart.france_large_domain_simulation.flwdir.png
@savefig user_guide.quickstart.large_domain_simulation.flwdir.png
plt.title("France - Flow direction");
.. ipython:: python
plt.imshow(mesh["flwacc"], norm=LogNorm());
plt.colorbar(label="Flow accumulation (m²)");
@savefig user_guide.quickstart.france_large_domain_simulation.flwacc.png
@savefig user_guide.quickstart.large_domain_simulation.flwacc.png
plt.title("France - Flow accumulation");
Then, we can initialize the `smash.Model` object
@@ -181,7 +142,7 @@ We can view the simulated discharge for one time step, for example the last one.
q = np.where(model.mesh.active_cell == 0, np.nan, q) # Remove the non-active cells from the plot
plt.imshow(q, norm=SymLogNorm(1e-4));
plt.colorbar(label="Discharge $(m^3/s)$");
@savefig user_guide.quickstart.france_large_domain_simulation.forward_run_q.png
@savefig user_guide.quickstart.large_domain_simulation.forward_run_q.png
plt.title("France - Discharge");
.. note::
@@ -222,11 +183,9 @@ specifying them in the ``return_options`` argument, for example only the two las
q = np.where(model.mesh.active_cell == 0, np.nan, q) # Remove the non-active cells from the plot
plt.imshow(q, norm=SymLogNorm(1e-4));
plt.colorbar(label="Discharge $(m^3/s)$");
@savefig user_guide.quickstart.france_large_domain_simulation.forward_run_q2.png
@savefig user_guide.quickstart.large_domain_simulation.forward_run_q2.png
plt.title("France - Discharge");
This concludes this second tutorial on `smash`.

.. ipython:: python
:suppress:
Loading

0 comments on commit a90791b

Please # to comment.