pyicon-core

In this section, the most important features of pyicon should be documented. However, we would like to mention that this documentation is not yet (and maybe will never be) exhaustive. Instead, we would like to refer to the many example notebooks where the most important use cases are introduced.

Common workflow using pyicon

The common workflow using pyicon typically consists of the following 5 steps:

1. Creating an IconData object

2. Loading the data

3. Doing some calculations

4. Interpolating the data

5. Plotting the data

6. (Optional) Saving the data to NetCDF

It is important to note that it is very well possible to use both, Jupyter Notebooks and simple python scripts.

Example of a typical use case

To be continued…

Creating an IconData object

Open data with xarray:

import xarray
ds = xr.open_dataset('/path/to/icon_data.nc')

Do some basic plotting:

ds.to.isel(depth=0, time=0).pyic.plot()

Regridding the data

In most cases it is not necessary to regrid the data to a regular grid. Most calculations are better done on the original grid. When it comes to plotting, pyicon, will intrinsically map to a regular grid befor plotting. Interpolating the data speeds up the plotting process a lot and often the loss of accuracy is tolerable in particular for high-resolution simulations. For getting optimal results, consider which resolution is necessary to have a figure on a screen or a paper with a resolution high enough to recognize all important details but try not to simply use the highest resolution since it usually creates unnecessary large figure sizes and computational effort. For global plots, e.g. often a resolution of 0.3deg is a good compromise between computational effort, figure size, and figure quality.

If in any case, if you need to remap the data manually, you can do this via:

da_rectgrid = pyic.interp_to_rectgrid_xr(ds.to.isel(depth=0, time=0))

The interpolation philosophy of pyicon is that interpolation should happen on the fly just before plotting. This means calculations should mostly be performed on the original grid and the final variable which should be plotted is only interpolated just before plotting. This interpolation is usually done in the computer memory and it is avoided to save the result to disk (although saving the interpolated data to disk is of course possible). With this approach, we avoid creating unnecessary interpolation files on disk, however, interpolation needs to be fast to get a smooth plotting workflow. To assure a fast interpolation, it is common in the usage of pyicon to first create interpolation files for a common source and target grids or vertical sections. These interpolation files are usually created only once and then re-used over and over again. This procedure allows for a very efficient interpolation even for large grids such as the SMT or R2B11 grid.

Create pre-defined interpolation files

For many typical sources and target grid combinations, there are already scripts that can generate interpolation files. These scripts can be found in:

pyicon/config_ckdtree

To generate the scripts enter the directory, open the desired script, adjust the necessary paths and grid information in the header and execute the script by (first line valid for mistral only):

source ../tools/conda_act_mistral_pyicon_env.sh
ipython --pylab
%run config_ckdtree_r2b6_oce_r0004.py

Creating the interpolation files can take quite some time (up to several hours for large grids like SMT or R2B11). Note that maybe other colleagues have already created interpolation files that fit your needs. It is perfectly fine to use those. They do not even to be copied just the path needs to be set appropriately as discussed below.

Create an own interpolation file

In most cases, it will not be necessary to create a new script for creating interpolation files, since there is already quite a list of example scripts for the most common grids. However, for more special source/target grids, it might be necessary to create a new interpolation script.

Therefore, go to:

cd pyicon/config_ckdtree

Copy one of the example scripts, e.g. config_ckdtree_r2b6_oce_r0004.py and edit as follows:

rev:          can be deleted
tgname:       how you want to call your grid (no spaces in the name, better rather short)
gname:        name of the grid file (see below how to find an appropriate grid file)
path_tgrid:   the path of the grid file
fname_tgrid:  keep as it is
path_ckdtree: path where the interpolation files should be saved, take any path which already exists
path_rgrid:   keep as it is
path_section: keep as it is

Usually, all ICON simulations need a grid file and you probably know which one was used for your simulation (a file containing lots of information about the horizontal grid). However, we only need a couple of variables most important clon, clan also sometimes important vlon, vlat, elon, elat. So you could use any file which contains these variables. In case you have a file that contains clon, clat but not vlon, vlat, elon, elat you can use this as well. However, you can only plot variables which are defined in the center (most of the variables, like ssh, pres, temp, u, v, w) but you cannot plot variables that are defined on vertices (vorticity) or edges (mass_flux). If you only want to derive interpolation indices for clon, clat you need to add load_egrid=False, and load_vgrid=False to all pyic.ckdtree_hgrid and pyic.ckdtree_section calls in the script.

Here is an example:

pyic.ckdtree_hgrid(lon_reg=[-180.,180.], lat_reg=[-90.,90.], res=1.0,
                  fname_tgrid  = fname_tgrid,
                  path_tgrid   = path_tgrid,
                  path_ckdtree = path_rgrid,
                  sname = sname,
                  gname = gname,
                  tgname = tgname,
                  load_egrid=False,
                  load_vgrid=False,
                  )

After modifying and executing the script, new interpolation files are generated for common target grids. If the target grids are not sufficient, it is possible to add custom target grids/sections by simply copying and adjusting the existing examples.

Using interpolation files

To use the interpolation files in scripts/notebooks, it is necessary to specify the path to the desired interpolation file. This file needs to be created for the source grid which corresponds to the data. In the following example, we assume a source grid R2B6 revision 4 for the ocean:

fpath_ckdtree = path_ckdtree + f'rectgrids/r2b6_oce_r0004_res0.30_180W-180E_90S-90N.npz'

Simply exchange the res0.30 by res0.10 to switch from a 0.3-degree target to a 0.1-degree target grid (of course both target grids need to be created beforehand as described above). Finally, you can do the interpolation by the following command using the above-defined interpolation file:

data_interpolated = pyic.interp_to_rectgrid(data, fpath_ckdtree, coordinates='clat clon')

Doing some calculations

Pyicon is designed to let the user concentrate on the actual derivations and manipulation of output data. Therefore many aspects like interpolating and plotting are encapsulated in ready-to-use functions that should facilitate the visualizations of the newly derived data. Regarding the computations themselves, pyicon supports different pyicon the usage of certain pyicon libraries like numpy and also to a lesser degree (for now) dask. Many standard derivations for the ocean and some for the atmosphere are already included in pyicon, however, the ultimate goal is that users are enabled to easily do their own calculations. In the following, some libraries do their own calculations and some pre-defined calculations are discussed.

Numpy computations

Typically all pyicon arrays are numpy arrays. Therefore, numpy is the easiest way of doing calculations within pyicon. However, when performance bottlenecks arise for very large data sets it might be advisable to use dask computations instead.

Dask computations

The support of dask in pyicon is still very experimental. More detailed documentation and examples will thus follow (hopefully) soon.

MPI4py computations

Sometimes, mpi4py can be used efficiently with pyicon to speed up repeating tasks by doing tasks in parallel e.g. along the time or vertical coordinate. A very common use case for mpi4py is for creating animations. Examples will follow (hopefully) soon.

Reconstructions

How to do typical ICON reconstructions (e.g. derive velocities at triangle centres from triangle edges) using the mimetic reconstructions defined in Korn (2017) can be found in the following notebook:

• examp_oce_reconstructions.ipynb

Some special diagnostics

We refer to the examples in the notebooks directory to see how some (and more) of the following diagnostics can be derived.

Ocean:

• overturning stream function

• barotropic stream function

• section transports

• vertical velocity

• tracer/heat fluxes

• zonal averaging

• horizontal and vertical velocity/tracer gradients

Atmosphere:

• height of isobar

• temperature on isobar

• vert. velocity conversion (omega to w)

• deriving density (equation of state)

• zonal averaging

• wind stress curl

Plotting the data

Examples for plotting ICON data can probably be found in every pyicon notebook. However, some particularly useful notebooks are:

• examp_intro_start.ipynb

• examp_oce_timeseries.ipynb

• examp_oceatm_crop_domain.ipynb

• examp_plotting_arrange_axes.ipynb

• examp_plotting_map_projections.ipynb

Saving data as NetCDF

The following notebook shows how saving data can be achieved:

• examp_oceatm_save_netcdf.ipynb