© Crown Copyright Met Office 2023
MONIO is not just an acronym. It is also so-named because it is single-threaded code (mon-; one, single), runs on a single PE in a full MPI context, and the main class is accessible via a singleton design pattern. It also happens that there is a lesser-known Star Wars character with the same name (https://starwars.fandom.com/wiki/Monio) 😐
MONIO has been written in C++17 to handle file I/O for the Met Office's (MO) JEDI (https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/) Data Assimilation (DA) model interfaces, but it is also developed with a gesture toward general applicability. MONIO represents an extendible set of classes to make reading and writing Network Common Data Form (NetCDF) files more friendly than using the bare NetCDF interface provided by Unidata alone.
MONIO provides a Monio
singleton class specifically for reading and writing state and increment, UGRID-format (http://ugrid-conventions.github.io/ugrid-conventions/) files generated by and to be consumed by the MO's "LFRic" model (https://www.metoffice.gov.uk/research/approach/modelling-systems/lfric). For this reason, it carries a few dependencies that are characteristic of any other JEDI-based repository.
MONIO is a CMake project. However, it is configured to make use of ECMWF's ECBuild (https://github.com/ecmwf/ecbuild). The CMakeLists.txt
file will need to modified if a dependency on ECBuild is to be avoided.
At its core, MONIO uses and depends upon Unidata's NetCDF-C++ (https://github.com/Unidata/netcdf-cxx4) library. This carries its own dependencies on Unidata's NetCDF-C (https://github.com/Unidata/netcdf-c) and the HDF Group's HDF5 (https://github.com/HDFGroup/hdf5) libraries, as well as some other supporting libraries.
Having been built for use with JEDI, MONIO also carries a dependency on ECMWF's Atlas (https://github.com/ecmwf/atlas) library. However, MONIO is written to compartmentalise this dependency in the AtlasReader
and AtlasWriter
classes. This puts users in a position to drop these classes to construct a NetCDF I/O solution without Atlas.
MONIO makes frequent use of JCSDA's OOPS (https://github.com/JCSDA/oops) oops::Log
tool and its util::DateTime
representation. Some light modifications could avoid these, but without modification MONIO will need to be compiled with a compatible version of OOPS.
As it is intended to run within an operational, meteorological DA context, MONIO is written to function with Message Passing Interface (MPI) communicators. JEDI wraps these with C++ code provided by ECMWF's ECKit (https://github.com/ecmwf/eckit), and MONIO adopts this approach. For this reason, MONIO also carries a dependency on ECKit. Which itself carries an underlying dependency on a suitable MPI implementation, e.g., Open MPI (https://www.open-mpi.org/software/).
After satisfying the dependencies outlined above, MONIO was built and tested using the Make and Ninja build systems, and the GCC and Intel compilers.
MONIO has been written to address the known use cases defined within the MO-JEDI context. These are captured in the public functons defined in the Monio
singleton class. Once Monio.h
is included in a source file it can be used directly.
In the following scenarios, the only common parameter is an atlas::FieldSet
that uses an atlas::CubedSphereGrid
and an atlas::Mesh
configured as a "cubedsphere_dual"
. Passing an atlas::FieldSet
of a different configuration can prevent MONIO from functioning correctly.
Most of the functions in monio::Monio
also take a reference to a std::vector<consts::FieldMetadata>
. This is a standard C++ std::vector
of consts::FieldMetadata
structs defined in MONIO's Constants.h
file. The struct instances in this vector must correspond to an atlas::Field
in the atlas::FieldSet
. They are accessed by the value of FieldMetadata.jediName
. If an atlas::Field
in the atlas::FieldSet
has no corresponding instance of consts::FieldMetadata
, they will not be processed. This may be an intended use-case. However, if an instance of consts::FieldMetadata
does not correspond to an atlas::Field
in the atlas::FieldSet
, an exeption will be generated and program execution will cease.
Reading of an LFRic-compatible, time-dependent, background file can be carried out with the following call:
monio::Monio::get().readState(localFieldSet, fieldMetadataVec, filePath, dateTime);
Where localFieldSet
is the atlas::FieldSet
to be populated with data from the file, fieldMetadataVec
is the std::vector<consts::FieldMetadata>
, filePath
is a std::string
defining a valid path to the file to be read, and dateTime
is an instance of util::DateTime
indicating what position in the time series data are required for.
Reading of an LFRic-compatible, time-independent, increment file can be carried out with the following call:
monio::Monio::get().readIncrements(localFieldSet, fieldMetadataVec, filePath);
Where localFieldSet
is the atlas::FieldSet
to be populated with data from the file, fieldMetadataVec
is the std::vector<consts::FieldMetadata>
, and filePath
is a std::string
defining a valid path to the file to be read.
Writing of an LFRic-compatible, time-independent, increment file is dependent on geometry data and other metadata being available at the resolution you intend to write. These will be available if MONIO has already been used to read LFRic-compatible data at the same resolution you intend to write (see the read functions described above). If MONIO has not been used for reading, writing will first require that geometry and metadata are copied from an appropriate input file using the following call:
monio::Monio::get().initialiseFile(grid, filePath);
Where grid
is a reference to the atlas::CubedSphereGrid
used to setup the geometry of the atlas::FieldSet
, filePath
is a std::string
defining a valid path to the file to be read. This particular function call returns an int
that determines the variable naming convention used in the input file. However, it is not necessary to capture and use that int
here.
Following this, or if MONIO was also used to read LFRic-compatible data, writing of an LFRic-compatible, time-independent, increment file can be carried out with the following call:
monio::Monio::get().writeIncrements(localFieldSet, fieldMetadataVec, filePath, isLFRicNaming);
Where localFieldSet
is the atlas::FieldSet
containing the data to be written to file, fieldMetadataVec
is the std::vector<consts::FieldMetadata>
, filePath
is a std::string
defining a valid path to the intended output file, and optionally, isLFRicNaming
is a bool
defining whether or not the variables should use LFRic or JEDI names. If this parameter is not defined, the variables will take the corresponding FieldMetadata.lfricWriteName
by default. The LFRic name is the only difference this function has with Monio::writeState
(below).
This method is intended for use with tests only. Writing of an LFRic-compatible, time-independent, state file is dependent on geometry data and other metadata being available at the resolution you intend to write. These will be available if MONIO has already been used to read LFRic-compatible data at the same resolution you intend to write (see the read functions described above). If MONIO has not been used for reading, writing will first require that geometry and metadata are copied from an appropriate input file using the following call:
monio::Monio::get().initialiseFile(grid, filePath);
Where grid
is a reference to the atlas::CubedSphereGrid
used to setup the geometry of the atlas::FieldSet
, filePath
is a std::string
defining a valid path to the file to be read. This particular function call returns an int
that determines the variable naming convention used in the input file. However, it is not necessary to capture and use that int
here.
Following this, writing of an LFRic-compatible, time-independent, state file can be carried out with the following call:
monio::Monio::get().writeState(localFieldSet, fieldMetadataVec, filePath, isLFRicNaming);
Where localFieldSet
is the atlas::FieldSet
containing the data to be written to file, fieldMetadataVec
is the std::vector<consts::FieldMetadata>
, filePath
is a std::string
defining a valid path to the intended output file, and optionally, isLFRicNaming
is a bool
defining whether or not the variables should use LFRic or JEDI names. If this parameter is not defined, the variables will take the corresponding FieldMetadata.lfricReadName
by default. The LFRic name is the only difference this function has with Monio::writeIncrements
(above).
For debugging, it may occasionally be useful to output an atlas::FieldSet
from any arbitrary position in the code into a NetCDF so that it can be examined. For this reason, MONIO offers the following call:
monio::Monio::get().writeFieldSet(localFieldSet, filePath);
Where localFieldSet
is the atlas::FieldSet
containing the data to be written to file, and filePath
is a std::string
defining a valid path to the intended output file. It should be noted that this particular writing method requires no existing geometric data or metadata to be provided, and so requires no supporting function call to be executed beforehand. For this reason, data are written in their default, Atlas order.
Any questions or issues can be raised on https://github.com/MetOffice/monio/issues or reported to philip.underwood@metoffice.gov.uk.