Intro to the ADF#

This package is meant to be an update/upgrade to the much used and beloved AMWG diagnostics package used by the atmospheric/chemistry CESM community. It is still under active development and is near it’s first major version.

What is it?#

The ADF is an open source, community developed Python-based set of collection of analysis (averaging), re-gridding, and plotting scripts aimed at replacing the old AMWG Diagnostics package (NCL-based). It is designed to be a multipurpose tool and has been built to be somewhat flexible for customization.

Who is it for?#

The ADF is geared towards users who are running CAM or CAM-like (MPAS) simulations that are looking to compare their runs agains other CAM simulations, observations/reanalysis or model comparison sets (like CMIP/AMIP).

Key Features#

 Flexible and Open Source

   - ADF code is completely open to the public
    - Users can modify their clone of the ADF to fit their needs if desired

 Use of GeoCAT functions (currently limited use)

 Use of YAML configuration files

   - helps to avoid changing source code

 Option for use of multiple processors

 Installation via Git and Conda package manager

   - CISL machines are set to run out of the box with required dependencies

 Centralize vertical interpolation

 -> Regridding and vertical interpolation script which interpolates all model variables with a vertical component onto a standard set of pressure levels

 -> Allows 3D model variable comparison against 3D observations

   - Assuming the observations are also on the same set of pressure levels

 Enable interpolation on MPAS vertical coordinate

 -> Checks for the MPAS height-based vertical coordinate, and if present it will enable pressure-to-pressure vertical interpolation required to get MPAS data onto the standard pressure levels used by the ADF

Types of ADF Comparisons#

There are essentially 3 types of comparisons:

  • CAM vs CAM

  • CAM vs Observations/Reanalysis

  • CAM vs CMIP

 Each of which can be run as:

  • Single Case Comparison - One test (experiment) case vs one baseline (control/target) case

  • Multiple Case Comparison** - Multiple test cases vs one baseline case

    •    ** In progress, will not be part of this tutorial at the moment :(

Each have their own requirements for the run-time config file. The requirements for each type of comparison will be addressed in their respective sections of this tutorial under the GUIDED EXAMPLES section.

Attention

Depending on how many years, plot types and variables you configure, the more time it takes

  • Rough average for all default plots, 10-20 years for 30 variables is ~45 minutes

Note

This is an active area of development; there are potential ways of cutting time/processes

ADF Basics#

Now let’s take a quick look at what the ADF actually does and the flow of it.

ADF Flow#

A simple look at the flow of the ADF
 ↳ Create time series files from monthly history files
  ↳ Create climatology files from either ADF generated or pre-existing time series files (ie CMIP)
   ↳ Regrid Test case from Baseline case and vice-versa from climatology files
    ↳ Run Analysis scripts
     ↳ Run Plotting scripts
      ↳ Generate Website pages

** Most parts of the ADF are optional and can be turned off for your desired need:

Potential Examples:

  • If you only want regridded data, turn off plotting, analysis, website parts

  • If you want plotting and only need a few plot images, you can turn off the website part

  • If you only want time series files, you can turn off all other parts

  • If you don’t care about the statistics (AMWG) tables, turn off analysis part

  • etc.

ADF Layout#

A simple look at the structure of the ADF directories

.
|-- config_amwg_default_plots.yaml
|-- config_cam_baseline_example.yaml
|-- env
|   `-- conda_environment.yaml
|-- jupyter_sample.ipynb
|-- lib
|   |-- adf_base.py
|   |-- adf_config.py
|   |-- adf_diag.py
|   |-- adf_info.py
|   |-- adf_obs.py
|   |-- adf_variable_defaults.yaml
|   |-- adf_web.py
|   |-- plotting_functions.py
|   |-- test
|   |   |-- pylintrc
|   |   `-- unit_tests
|   |       |-- pytest.ini
|   |       |-- test_adf_base.py
|   |       |-- test_adf_config.py
|   |       `-- test_files
|   |           |-- config_cam_double_nested.yaml
|   |           |-- config_cam_keywords.yaml
|   |           `-- config_cam_unset_var.yaml
|   `-- website_templates
|       |-- adf_diag.css
|       |-- NCAR.gif
|       |-- template.html
|       |-- template_index.html
|       |-- template_mean_diag.html
|       |-- template_mean_tables.html
|       |-- template_multi_case_index.html
|       |-- template_table.html
|       `-- template_var.html
|-- LICENSE
|-- README.md
|-- run_adf_diag
`-- scripts
    |-- analysis
    |   `-- amwg_table.py
    |-- averaging
    |   `-- create_climo_files.py
    |-- plotting
    |   |-- cam_taylor_diagram.py
    |   |-- global_latlon_map.py
    |   |-- global_latlon_vect_map.py
    |   |-- meridional_mean.py
    |   |-- polar_map.py
    |   |-- qbo.py
    |   |-- regional_map_multicase.py
    |   `-- zonal_mean.py
    `-- regridding
        `-- regrid_and_vert_interp.py

ADF Setup#

A simple look at the steps for using the ADF

0. Run CAM/other simulation to get ADF input files
   history files (ie: h0, h1, etc.)
   time series files (ie: C/AMIP)

1. Configure copies of yaml file(s) for the input files
   config_cam_baseline_example.yaml
   lib/adf_variable_defaults.yaml

2. Run the ADF
   simple command line call: ./run_adf_diag config_myadf.yaml    run ADF in Jupyter notebook

3. View Diagnostics
   Locally
   Jupyter/JupyterHub
   Publish to website