Run Time - config_cam_baseline_example.yaml#

Introduction#

This is probably the most important file for the ADF. This stores all the necessary information that the ADF needs to run as well as all the relevant information about the case and baseline/observation/cmip runs.

Each section of the yaml file represent grouped subsets of information sent to the ADF and each have variable names that get fed into the ADF in a specific fashion. In this yaml file variables can be set and called with the ${} syntax.

IE if we set a user variable user: richling we can call this by ${user} and richling wil be used where ever it is referenced.

BIG CAVEAT:
In this particular yaml file, we have built subsections in which variables set in a subsection are only callable by referencing the subsection first with a “.” then variable name.

IE: Say we want to set the climo years in the test case subsection `diag_cam_climo` and call them in the baseline subsection `diag_cam_baseline_climo`

diag_cam_climo:

start_year: 1995 -> we would reference this with diag_cam_climo.start_year
end_year: 2000 -> we would reference this with diag_cam_climo.end_year

Now if we wanted to reference these climo years in the baseline subsection:

diag_cam_baseline_climo:

start_year: ${diag_cam_climo.start_year}
end_year: ${diag_cam_climo.end_year}

Warning

Please do not modify this file!

It is recommended to make a copy of this file, make modifications in that copy, and then run that with the ADF.

Example

From the ADF root directory

cp config_cam_example.yaml config_your_run.yaml

Then do your editing in config_your_run.yaml

Nested listed variables: Variables cannot be used in nested varaibles lists. The multi-case setup is one example of nested listign for variables.

** does not work **

user: richling
case: some_run
some_path:
- /glade/u/home/${user}
- /glade/u/home/${user}/${case}

** works **

some_path:
- /glade/u/home/richling
- /glade/u/home/richling/some_run

Subsections#

The subsections that may need to be customized for your run are:

diag_basic_info
- Basic overall run variables
diag_cam_climo
- Test (experiment) case variables
diag_cam_baseline_climo
- Baseline (control) case variables
- Note: these are the same variables as diag_cam_climo just for the baseline case
diag_cvdp_info
- For running the Climate Variability Diagnostics Package https://www.cesm.ucar.edu/projects/cvdp
- This will run in the background of the ADF
time_averaging_scripts
- Climotology file creation
regridding_scripts
- Regridding of climo files
analysis_scripts
- AMWG statisics tables
plotting_scripts
- Various plotting scripts
diag_var_list
- List of CAM variables for ADF to run

Warning

If variable strings start with a zero, the quantity must be in quotes in this yaml file.

example: desired nick name 026b

case_nickname: "026b" (notice the quotes for a leading zero)

diag_basic_info#

`compare_obs`#

Is this a model vs observations comparison? - true/false

If false or missing, then a model-model comparison is assumed

`obs_data_loc`#

Location of observational datasets. NOTE: you can override this on a variable by variable basis in the variable defaults file

Note: this only matters if compare_obs is true and the path isn't specified in the variable defaults file.
default: /glade/work/nusbaume/SE_projects/model_diagnostics/ADF_obs
example: /path/to/your/obs/files/

`cam_regrid_loc`#

Location where re-gridded and interpolated CAM climatology files are stored

Required if regridding is desired and

example: /glade/scratch/${user}/ADF/regrid

`cam_overwrite_regrid`#

Overwrite CAM re-gridded files? - true/false

If false, or missing, then regridding will be skipped for regridded variables that already exist in cam_regrid_loc

`cam_diag_plot_loc`#

Location where diagnostic plots are stored
The ADF will create a directory for the plots in this directory in the form of {cam_case_name}_{YYYY}_{YYYY}_vs_{baseline_name}_{YYYY}_{YYYY}

example: /glade/scratch/${user}/ADF/plots

`hist_num`#

CAM history file number to use (h0, h1, h2, etc)

Currently only affects timeseries generation, as everything else uses the timeseries files themselves.
If this variable is not present then it will default to "cam.h0"
example: cam.h4

`defaults_file`#

Location of ADF variable plotting defaults YAML file if not using ADF default file: lib/adf_variable_defaults.yaml

example: /path/to/custom/defaults/file/my_variable_defaults.yaml

`plot_press_levels`#

Vertical pressure levels (in hPa) on which to plot 3-D variables when using horizontal (e.g. lat/lon) map projections

If this config option is missing, then no 3-D variables will be plotted on horizontal maps. Please note too that pressure levels must currently match what is available in the observations file in order to be plotted in a model vs obs run:
example: [200,850]

`central_longitude`#

Longitude line on which to center all lat/lon maps

If missing, then the central longitude will default to 180 degrees E.
example: 0

`num_procs`#

Number of processors on which to run the ADF

If this config variable isn't present then the ADF defaults to one processor.
If you set it to * then it will default to all of the processors available on a single node/machine
default: 8

`redo_plot`#

Remake plots? - true/false

If set to true, redo all plots even if they already exist
If set to false, a plot will be skipped if found
default: false

`create_html`#

Generate HTML website - true/false

If false or missing, no html pages will be made
If true The website files themselves will be located in the path specified by cam_diag_plot_loc, under the {diag_run}/website subdirectory, where {diag_run} is the subdirectory created for this particular diagnostics run (usually {case}_YYYY_YYYY_vs_obs or {case}_YYYY_YYYY_vs_{baseline}_YYYY_YYYY)

diag_cam_climo#

`calc_cam_climo`#

Calculate climatology files? - true/false

If false, the climatology files will not be created
default: true

`cam_overwrite_climo`#

Overwrite CAM climatology files? - true/false

If false, or not present, then already existing climatology files will be skipped
default: false

`cam_case_name - Required `#

** Required **

Name of CAM run

example: b.e20.BHIST.f09_g17.20thC.297_05

`case_nickname`#

Case nickname to be used for plotting purposes as well as any custom place in the diagnsotics output if desired

If left blank, it will default to cam_case_name
example: cool_nickname

`cam_hist_loc`#

** Required **

Location of CAM history (h0) files

example: /glade/p/cesm/ADF/${diag_cam_climo.cam_case_name}

`cam_climo_loc`#

Location of CAM climatologies (to be created and then used by this script)

example: /glade/scratch/${user}/ADF/${diag_cam_climo.cam_case_name}/climo

`start_year`#

Model year when time series files should start

Note: Leaving this entry blank will make time series start at earliest available year
example: 1990
example: 10
example: 0010??

`end_year`#

Model year when time series files should end

Note: Leaving this entry blank will make time series end at latest available year
example: 1999
example: 15

`cam_ts_done`#

Do time series files need to be generated? - true/false

If true, then diagnostics assumes that model files are already time series
If false, or if simply not present, then diagnostics will attempt to create time series files from history (time-slice) files
default: false

`cam_ts_save`#

Save interim time series files? - true/false

Warning

If true, this can take up a significant amount of space

default: true

`cam_overwrite_ts`#

Overwrite time series files, if found? - true/false

If set to false, then time series creation will be skipped if files are found
default: false

`cam_ts_loc`#

Location where time series files are (or will be) stored

example: /glade/scratch/${user}/ADF/${diag_cam_climo.cam_case_name}/ts

Warning

If this is a CMIP-like case, this should be set to the same location as cam_hist_loc

diag_cam_baseline_climo#

Note

These are the same variable names as diag_cam_climo above, but for the baseline case

They are not necessary if comapring against observations, and will be skipped

`calc_cam_climo`#

Calculate climatology files? - true/false

If false, the climatology files will not be created
default: true

`cam_overwrite_climo`#

Overwrite CAM climatology files? - true/false

If false, or not present, then already existing climatology files will be skipped
default: false

`cam_case_name`#

** Required **

Name of baseline case (or CAM/other run name)

example: b.e20.BHIST.f09_g16.20thC.125.02

`case_nickname`#

Baseline nickname to be used for plotting purposes as well as any custom place in the diagnsotics output if desired

If left blank, it will default to cam_case_name
example: cool_nickname

`cam_hist_loc`#

** Required **

Location of baseline history (h0) files

example: /glade/p/cesm/ADF/${diag_cam_baseline_climo.cam_case_name}

`cam_climo_loc`#

Location of baseline climatologies (to be created and then used by this script)

example: /glade/scratch/${user}/ADF/${diag_cam_baseline_climo.cam_case_name}/climo

`start_year`#

Baseline year when time series files should start

Note: Leaving this entry blank will make time series start at earliest available year
example: 1990
example: 10
example: 0010??

`end_year`#

Baseline year when time series files should end

Note: Leaving this entry blank will make time series end at latest available year
example: 1999
example: 15

`cam_ts_done`#

Do time series files need to be generated? - true/false

If true, then diagnostics assumes that baseline files are already time series
If false, or if simply not present, then diagnostics will attempt to create time series files from history (time-slice) files
default: false

`cam_ts_save`#

Save interim time series files? - true/false

Warning

If true, this can take up a significant amount of space

default: true

`cam_overwrite_ts`#

Overwrite time series files, if found? - true/false

If set to false, then time series creation will be skipped if files are found
default: false

`cam_ts_loc`#

Location where time series files are (or will be) stored

example: /glade/scratch/${user}/ADF/${diag_cam_baseline_climo.cam_case_name}/ts

Warning

If this is a CMIP-like case, this should be set to the same location as cam_hist_loc

diag_cvdp_info#

`cvdp_run`#

Run the CVDP in the background? - true/false

default: false

`cvdp_codebase_loc`#

Set location of CVDP codebase

CGD systems path = /home/asphilli/CESM-diagnostics/CVDP/Release/v5.2.0/
CISL systems path = /glade/u/home/asphilli/CESM-diagnostics/CVDP/Release/v5.2.0/
github location = https://github.com/NCAR/CVDP-ncl
default: /glade/u/home/asphilli/CESM-diagnostics/CVDP/Release/v5.2.0/

`cvdp_loc`#

Location where the CVDP files will be stored

example: /glade/scratch/${user}/cvdp/

Diagnostic Scripts “Subsections”#

Each of the following sets are also considered subsections. These are all components of the actual diagnostics workflow of the ADF.

time_averaging_scripts#

This section allows for any time averaging scripts

Must be located in scripts/averaging within the ADF root directory:

scripts/averaging/create_climo_files.py
scripts/averaging/create_TEM_files.py

In this yaml file they would be listed as:

time_averaging_scripts:
- create_climo_files
- create_TEM_files

regridding_scripts#

This section allows for any regridding scripts

Must be located in scripts/regridding!

- scripts/regridding/regrid_and_vert_interp.py

In this yaml file they would be listed as:

regridding_scripts:
- create_climo_files

analysis_scripts#

This section allows for a broad spectra of data analysis scripts

Must be located in scripts/analysis!

- scripts/analysis/amwg_table.py

Possible addition examples:
- scripts/analysis/amwg_chem_table.py
- scripts/analysis/amwg_aerosol_table.py

In this yaml file they would be listed as:

analysis_scripts:
- amwg_table
Possible addition examples:
- amwg_chem_table
- amwg_aerosol_table

plotting_scripts#

Set of plotting scripts, both ADF default as well as any custom scripts

Must be located in scripts/plotting!
- scripts/plotting/cam_taylor_diagram.py
- scripts/plotting/global_latlon_map.py
- scripts/plotting/…
Possible addition examples:
- scripts/plotting/time_series.py

In this yaml file they would be listed as:

plotting_scripts:
- global_latlon_map
- zonal_mean
- meridional_mean
- polar_map
- global_latlon_vect_map
- cam_taylor_diagram
- qbo
- tem -> requires more setup info with tem_info
- regional_map_multicase -> requires more setup info with region_multicase

diag_var_list#

List of variables to be executed in the ADF

These usually are the CAM variables, but nocesessarily the only ones, ie RESTOM can be added to this list and the ADF will attempt to calculate based off constituents.

If the ADF searches for time series/climo/regridded/tem, etc. files for a given variable from this list and does not find one in the data files, it will skip that variable with an explaination in most cases. The ADF will not crash, but continue on to the next variable in the list.

tem_info#

Options for TEM diagnostics (./averaging/create_TEM_files.py and ./plotting/temp.py)
tem_info:
#Location where TEM files are stored:
#If path not specified or commented out, skip TEM calculation
tem_loc: /glade/scratch/richling/adf-output/ADF-data/TEM/TEST/
#TEM history file number
hist_num: h4

#Overwrite TEM files, if found?
overwrite_tem_case: false

#For multiple test cases
#overwrite_tem_case
# - false
# - true

overwrite_tem_base: false

ADF Tutorial

Run Time - config_cam_baseline_example.yaml

Contents

Run Time - config_cam_baseline_example.yaml#

Introduction#

Subsections#

diag_basic_info#

compare_obs#

obs_data_loc#

cam_regrid_loc#

cam_overwrite_regrid#

cam_diag_plot_loc#

hist_num#

defaults_file#

plot_press_levels#

central_longitude#

num_procs#

redo_plot#

create_html#

diag_cam_climo#

calc_cam_climo#

cam_overwrite_climo#

cam_case_name - ** Required **#

case_nickname#

cam_hist_loc#

cam_climo_loc#

start_year#

end_year#

cam_ts_done#

cam_ts_save#

cam_overwrite_ts#

cam_ts_loc#

diag_cam_baseline_climo#

calc_cam_climo#

cam_overwrite_climo#

cam_case_name#

case_nickname#

cam_hist_loc#

cam_climo_loc#

start_year#

end_year#

cam_ts_done#

cam_ts_save#

cam_overwrite_ts#

cam_ts_loc#

diag_cvdp_info#

cvdp_run#

cvdp_codebase_loc#

cvdp_loc#

Diagnostic Scripts “Subsections”#

time_averaging_scripts#

regridding_scripts#

analysis_scripts#

plotting_scripts#

diag_var_list#

tem_info#

region_multicase#

`compare_obs`#

`obs_data_loc`#

`cam_regrid_loc`#

`cam_overwrite_regrid`#

`cam_diag_plot_loc`#

`hist_num`#

`defaults_file`#

`plot_press_levels`#

`central_longitude`#

`num_procs`#

`redo_plot`#

`create_html`#

`calc_cam_climo`#

`cam_overwrite_climo`#

`cam_case_name - Required `#

`case_nickname`#

`cam_hist_loc`#

`cam_climo_loc`#

`start_year`#

`end_year`#

`cam_ts_done`#

`cam_ts_save`#

`cam_overwrite_ts`#

`cam_ts_loc`#

`calc_cam_climo`#

`cam_overwrite_climo`#

`cam_case_name`#

`case_nickname`#

`cam_hist_loc`#

`cam_climo_loc`#

`start_year`#

`end_year`#

`cam_ts_done`#

`cam_ts_save`#

`cam_overwrite_ts`#

`cam_ts_loc`#

`cvdp_run`#

`cvdp_codebase_loc`#

`cvdp_loc`#