{ "cells": [ { "cell_type": "markdown", "id": "d88a8f2e-7cc7-4664-88b8-aaf7d1a0a555", "metadata": { "tags": [] }, "source": [ "# ADF in Jupyter\n", "\n", "Now that we have seen how to run the ADF via terminal, let's take a look at how to run the ADF through Jupyter.\n", "\n", "### Why Jupyter?\n", "\n", "If you are looking to use the ADF in a more developmental or interactive way, using Jupyter will offer a lot more flexibility than running through the terminal.\n", "\n", "One major advantage of this way of running the ADF will allow the user to immediately have access to output data in a Jupyter notebook which can allow for further development or use of diagnostics that aren't in the ADF by defualt.\n", "\n", "One scenario to highlight this flexibility could be for tuning specific parameters of your model run that need a specific diagnostics set of tools that maybe only good for this tune. Now the user can build custom diagnostics code within the same Jupyter notebook that the ADF was run in!\n", "\n", "If you need help with navigating Jupyter Nobeooks or NCAR's JupyterHub, please see the reference section of this tutorial." ] }, { "cell_type": "markdown", "id": "a50566c5-005a-45f2-afa1-b00d1a20defb", "metadata": {}, "source": [ "### Step 1: Run ADF\n", "\n", "This will simulate running the ADF in the terminal, so nothing will be different here. In order to run the ADF, any of the NPL conda environments will suffice in Jupyter. NCAR has a couple to choose from (as of this tutorial):\n", "\n", " NPL 2023a\n", " NPL 2023b" ] }, { "cell_type": "markdown", "id": "91f6ab33-f72c-4e07-9c73-608f9064b687", "metadata": {}, "source": [ "If you are not up tp date or familiar with JupyterHub/Jupyter Notebooks, to change the kernel (environment) from the default Python3 (or None) to a different kerenel, at the top right of the notebook is an option to switch kernels:\n", "\n", "![kernel location](kernel_location.png)\n", "\n", "Click on the kernel and it will bring up a dropdown box with available kernels:\n", "\n", "![kernel dropdown](kernel_dropdown.png)\n", "\n", "Find one of the NPL environments and select that:\n", "\n", "NOTE: your available kernels may not be same as the ones displayed here.\n", "\n", "![kernel options](kernel_options.png)" ] }, { "cell_type": "raw", "id": "053799ae-8b20-4b9e-b6ca-63f257f60791", "metadata": {}, "source": [ "\n", " \n", "

Using the download attribute of a tag to create file download button using JavaScript.

\n", "

Click the below button to download this Jupyter Notebook

\n", " \n", " \n", " \n", "
\n", " \n", "" ] }, { "cell_type": "code", "execution_count": 3, "id": "98d13334-1c5f-4faf-9027-0069b8d7582c", "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "\n", "import os.path\n", "from pathlib import Path\n", "import sys" ] }, { "cell_type": "code", "execution_count": 4, "id": "90403315-d11e-4f22-9158-481b4cf7ae14", "metadata": {}, "outputs": [], "source": [ "# Determine ADF directory path\n", "# If it is in your cwd, set adf_path = local_path, \n", "# otherwise set adf_path appropriately\n", "\n", "local_path = os.path.abspath('')" ] }, { "cell_type": "code", "execution_count": 5, "id": "8e0b805b-c12e-4e8a-a80c-16ea535994ab", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "current working directory = /glade/work/richling/ADF/adf-demo/notebooks/basic_examples\n", "ADF path = /glade/work/richling/ADF/ADF/\n" ] } ], "source": [ "#adf_path = \"/glade/work/{user}/ADF\" # <-- uncomment and use your username\n", "adf_path = \"/glade/work/richling/ADF/ADF/\" # <-- then comment out\n", "\n", "print(f\"current working directory = {local_path}\")\n", "print(f\"ADF path = {adf_path}\")" ] }, { "cell_type": "code", "execution_count": 6, "id": "aca1da90-a5ea-45f4-bb59-f81f256249b5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The lib scripts live here, right? /glade/work/richling/ADF/ADF/lib\n", "The plotting scripts live here, right? /glade/work/richling/ADF/ADF/scripts/plotting\n" ] } ], "source": [ "#set path to ADF lib\n", "lib_path = os.path.join(adf_path,\"lib\")\n", "print(f\"The lib scripts live here, right? {lib_path}\")\n", "\n", "#set path to ADF plotting scripts directory\n", "plotting_scripts_path = os.path.join(adf_path,\"scripts\",\"plotting\")\n", "print(f\"The plotting scripts live here, right? {plotting_scripts_path}\")\n", "\n", "#Add paths to python path:\n", "sys.path.append(lib_path)\n", "sys.path.append(plotting_scripts_path)" ] }, { "cell_type": "code", "execution_count": 7, "id": "68ecdbc8-8d1d-48c8-83e6-1f16b334068b", "metadata": {}, "outputs": [], "source": [ "#import ADF diagnostics object\n", "from adf_diag import AdfDiag\n", "\n", "# If this fails, check your paths output in the cells above,\n", "# and that you are running the NPL (conda) Kernel\n", "# You can see all the paths being examined by un-commenting the following:\n", "#sys.path" ] }, { "cell_type": "markdown", "id": "85b66af4-694d-4ee5-be1f-99d2671d0377", "metadata": {}, "source": [ "##### Load the run-time config file into the ADF module\n", "\n", "This will create the ADF python object" ] }, { "cell_type": "code", "execution_count": 8, "id": "6ca43a0e-b22e-42c0-ba66-72e92699c84a", "metadata": {}, "outputs": [], "source": [ "# Set path for config YAML file\n", "#config_path = \"/path/to/your/yaml/file/\"\n", "config_path = \"/glade/work/richling/ADF/adf-demo/config_files/\"\n", "\n", "# Set name of config YAML file:\n", "config_fil_str = \"config_single_model_vs_model.yaml\"\n", "\n", "# Make full path to config file\n", "config_file=os.path.join(config_path,config_fil_str)" ] }, { "cell_type": "code", "execution_count": 9, "id": "a7296af2-e7d2-44f4-83ec-87da1831c7ea", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#Initialize ADF object with config file\n", "adf = AdfDiag(config_file)\n", "adf" ] }, { "cell_type": "markdown", "id": "d3539201-f040-4016-9044-499c7e6cfd08", "metadata": {}, "source": [ "The following functions are what get called under the hood when the ADF is run through the terminal" ] }, { "cell_type": "markdown", "id": "6b552f4c-ab32-4d66-bcc2-88bd7653c6dd", "metadata": {}, "source": [ "##### Create Time Series Files\n", "\n", "Create model time series" ] }, { "cell_type": "code", "execution_count": null, "id": "3bf1f4ed-ef44-470d-8407-d5742b4da272", "metadata": {}, "outputs": [], "source": [ "%%time\n", "adf.create_time_series()" ] }, { "cell_type": "markdown", "id": "58ef4b9c-146b-44ce-bdcf-9091531ac55e", "metadata": {}, "source": [ "Output:" ] }, { "cell_type": "markdown", "id": "cf63a7fd-5868-44de-ac26-3844a5e1412f", "metadata": {}, "source": [ " Generating CAM time series files...\n", " Processing time series for case 'f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001' :\n", " - time series for CLDHGH\n", " - time series for CLDICE\n", " Adding PS to file\n", " - time series for CLDLIQ\n", " Adding PS to file\n", " - time series for CLDLOW\n", " - time series for CLDMED\n", " - time series for CLDTOT\n", " - time series for CLOUD\n", " Adding PS to file\n", " - time series for FLNS\n", " - time series for FLNT\n", " - time series for FLNTC\n", " - time series for FSNS\n", " - time series for FSNT\n", " - time series for FSNTC\n", " - time series for LHFLX\n", " - time series for LWCF\n", " - time series for OMEGA500\n", " - time series for PBLH\n", " - time series for PRECL\n", " - time series for PRECT\n", " - time series for PRECSL\n", " - time series for PRECSC\n", " - time series for PRECSC\n", " - time series for PRECC\n", " - time series for PS\n", " - time series for PSL\n", " - time series for QFLX\n", " - time series for Q\n", " Adding PS to file\n", " - time series for RELHUM\n", " Adding PS to file\n", " - time series for SHFLX\n", " - time series for SST\n", " - time series for SWCF\n", " - time series for T\n", " Adding PS to file\n", " - time series for TAUX\n", " - time series for TAUY\n", " - time series for TGCLDIWP\n", " - time series for TGCLDLWP\n", " - time series for TMQ\n", " - time series for TREFHT\n", " - time series for TS\n", " - time series for U\n", " Adding PS to file\n", " - time series for U10\n", " - time series for ICEFRAC\n", " - time series for OCNFRAC\n", " - time series for LANDFRAC\n", " ...CAM time series file generation has finished successfully.\n", " CPU times: user 256 ms, sys: 40.5 ms, total: 297 ms\n", " Wall time: 1min 5s" ] }, { "cell_type": "code", "execution_count": null, "id": "3194e320-0384-47e8-a8f2-d08cf2032cec", "metadata": {}, "outputs": [], "source": [ "%%time\n", "#Create model baseline time series (if needed):\n", "\n", "# Since we are doing model vs model\n", "if not adf.compare_obs:\n", " adf.create_time_series(baseline=True)" ] }, { "cell_type": "markdown", "id": "e45ad6f7-0e33-4c1e-aa6c-39d4bd4d70d4", "metadata": {}, "source": [ "Output:" ] }, { "cell_type": "markdown", "id": "d78f90e2-81ec-415e-b54b-57a551c6c7f2", "metadata": {}, "source": [ " Generating CAM time series files...\n", " Processing time series for case 'f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001' :\n", " - time series for CLDHGH\n", " - time series for CLDICE\n", " Adding PS to file\n", " - time series for CLDLIQ\n", " Adding PS to file\n", " - time series for CLDLOW\n", " - time series for CLDMED\n", " - time series for CLDTOT\n", " - time series for CLOUD\n", " Adding PS to file\n", " - time series for FLNS\n", " - time series for FLNT\n", " - time series for FLNTC\n", " - time series for FSNS\n", " - time series for FSNT\n", " - time series for FSNTC\n", " - time series for LHFLX\n", " - time series for LWCF\n", " - time series for OMEGA500\n", " - time series for PBLH\n", " - time series for PRECL\n", " - time series for PRECT\n", " - time series for PRECSL\n", " - time series for PRECSC\n", " - time series for PRECSC\n", " - time series for PRECC\n", " - time series for PS\n", " - time series for PSL\n", " - time series for QFLX\n", " - time series for Q\n", " Adding PS to file\n", " - time series for RELHUM\n", " Adding PS to file\n", " - time series for SHFLX\n", " - time series for SST\n", " - time series for SWCF\n", " - time series for T\n", " Adding PS to file\n", " - time series for TAUX\n", " - time series for TAUY\n", " - time series for TGCLDIWP\n", " - time series for TGCLDLWP\n", " - time series for TMQ\n", " - time series for TREFHT\n", " - time series for TS\n", " - time series for U\n", " Adding PS to file\n", " - time series for U10\n", " - time series for ICEFRAC\n", " - time series for OCNFRAC\n", " - time series for LANDFRAC\n", " ...CAM time series file generation has finished successfully.\n", " CPU times: user 180 ms, sys: 141 ms, total: 321 ms\n", " Wall time: 1min 6s" ] }, { "cell_type": "markdown", "id": "32926e0d-0715-415e-aed3-ff4bd0a1a00f", "metadata": {}, "source": [ "##### Create Climo Files\n", "\n", "This will run the creation of both test and baseline cases" ] }, { "cell_type": "code", "execution_count": null, "id": "e97c9567-e0d4-4925-b936-571543fd6f84", "metadata": {}, "outputs": [], "source": [ "%%time\n", "#Create model climatology (climo) files.\n", "adf.create_climo()" ] }, { "cell_type": "markdown", "id": "e036b2f5-5182-4580-921e-eea39adcb6ff", "metadata": {}, "source": [ "Output:" ] }, { "cell_type": "markdown", "id": "0ce0aeb4-ee22-4a11-944c-f6dd82a3b6ad", "metadata": {}, "source": [ " Calculating CAM climatologies...\n", " Calculating climatologies for case 'f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001' :\n", " /glade/scratch/richling/ADF/adf_tutorial/data/climo/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001 not found, making new directory\n", " Calculating climatologies for case 'f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001' :\n", " /glade/scratch/richling/ADF/adf_tutorial/data/climo/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001 not found, making new directory\n", " ...CAM climatologies have been calculated successfully.\n", " CPU times: user 226 ms, sys: 307 ms, total: 533 ms\n", " Wall time: 5min 13s" ] }, { "cell_type": "markdown", "id": "acc6e9d6-03d8-446b-9659-87670874a660", "metadata": {}, "source": [ "##### Regrdding Climo Files\n", "\n", "Regrid model climatology files to match either observations or CAM baseline climatologies.\n", "\n", "This call uses the `regridding_scripts` specified in the run-time config file" ] }, { "cell_type": "code", "execution_count": null, "id": "6816e9c2-2c4a-4320-bf6a-32c5ed50695b", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "15565a68-8056-4815-a898-c2fff9a54fcb", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "33d7dce5-8160-48e8-a147-998e8882c2e8", "metadata": {}, "outputs": [], "source": [ "%%time\n", "adf.regrid_climo()" ] }, { "cell_type": "markdown", "id": "07f208c0-b879-42ce-8315-a812ed3a56de", "metadata": {}, "source": [ "Output:" ] }, { "cell_type": "markdown", "id": "2d6d4025-60fe-420b-b70d-8e39de4c9080", "metadata": {}, "source": [ " Regridding CAM climatologies...\n", " /glade/scratch/richling/ADF/adf_tutorial/data/regrid not found, making new directory\n", " Regridding case 'f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001' :\n", " - regridding PS (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding LANDFRAC (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding OCNFRAC (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding CLDHGH (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding CLDICE (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " - regridding CLDLIQ (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " - regridding CLDLOW (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding CLDMED (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding CLDTOT (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding CLOUD (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " - regridding FLNS (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding FLNT (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding FLNTC (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding FSNS (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding FSNT (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding FSNTC (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding LHFLX (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding LWCF (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding OMEGA500 (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding PBLH (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding PRECL (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding PRECT (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding PRECSL (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding PRECSC (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding PRECSC (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " Regridded file already exists, so skipping...\n", " - regridding PRECC (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding PSL (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding QFLX (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding Q (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " - regridding RELHUM (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " - regridding SHFLX (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding SST (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding SWCF (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding T (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " - regridding TAUX (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding TAUY (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding TGCLDIWP (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding TGCLDLWP (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding TMQ (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding TREFHT (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding TS (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding U (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " Please ignore the interpolation warnings that follow!\n", " Interpolation point out of data bounds encountered\n", " - regridding U10 (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " - regridding ICEFRAC (known targets: ['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001'])\n", " ...CAM climatologies have been regridded successfully.\n", " CPU times: user 1min 42s, sys: 9.85 s, total: 1min 52s\n", " Wall time: 2min 11s" ] }, { "cell_type": "code", "execution_count": null, "id": "c81eeb8d-751e-4453-80e7-33aeeb5d626f", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "8cbf551b-1dfb-465c-8e95-dd1b38d88ea7", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "ce434d15-8719-4cc8-a9cd-914b44e06e15", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "524c946d-6b1d-4347-8aba-22d201b3c187", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "e3b8f6f0-97d2-4a69-9df0-efa3663b85d4", "metadata": {}, "source": [ "##### Run Analysis Scripts\n", "\n", "Perform analyses on the simulation(s).\n", "This call uses the `analysis_scripts` specified in the run-time config file" ] }, { "cell_type": "code", "execution_count": null, "id": "4ebe3254-1d30-4f41-af3e-8615d69c6e2d", "metadata": {}, "outputs": [], "source": [ "adf.perform_analyses()" ] }, { "cell_type": "markdown", "id": "75431df2-5265-42e2-9b9f-e4ea715ed10e", "metadata": {}, "source": [ "Output:" ] }, { "cell_type": "markdown", "id": "7c72800c-f069-412e-9b11-7faca77323a1", "metadata": {}, "source": [ " Calculating AMWG variable table...\n", " /glade/scratch/richling/ADF/adf_tutorial/plots/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001_1995_2000_vs_f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_1995_2000 not found, making new directory\n", " - Variable 'LANDFRAC' being added to table\n", " - Variable 'OCNFRAC' being added to table\n", " - Variable 'CLDHGH' being added to table\n", " - Variable 'CLDICE' being added to table\n", " Variable 'CLDICE' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'CLDLIQ' being added to table\n", " Variable 'CLDLIQ' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'CLDLOW' being added to table\n", " - Variable 'CLDMED' being added to table\n", " - Variable 'CLDTOT' being added to table\n", " - Variable 'CLOUD' being added to table\n", " Variable 'CLOUD' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'FLNS' being added to table\n", " - Variable 'FLNT' being added to table\n", " - Variable 'FLNTC' being added to table\n", " - Variable 'FSNS' being added to table\n", " - Variable 'FSNT' being added to table\n", " - Variable 'FSNTC' being added to table\n", " - Variable 'LHFLX' being added to table\n", " - Variable 'LWCF' being added to table\n", " - Variable 'OMEGA500' being added to table\n", " - Variable 'PBLH' being added to table\n", " - Variable 'PRECL' being added to table\n", " - Variable 'PRECT' being added to table\n", " - Variable 'PRECSL' being added to table\n", " - Variable 'PRECSC' being added to table\n", " - Variable 'PRECSC' being added to table\n", " - Variable 'PRECC' being added to table\n", " - Variable 'PS' being added to table\n", " - Variable 'PSL' being added to table\n", " - Variable 'QFLX' being added to table\n", " - Variable 'Q' being added to table\n", " Variable 'Q' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'RELHUM' being added to table\n", " Variable 'RELHUM' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'SHFLX' being added to table\n", " - Variable 'SST' being added to table\n", " - Variable 'SWCF' being added to table\n", " - Variable 'T' being added to table\n", " Variable 'T' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'TAUX' being added to table\n", " - Variable 'TAUY' being added to table\n", " - Variable 'TGCLDIWP' being added to table\n", " - Variable 'TGCLDLWP' being added to table\n", " - Variable 'TMQ' being added to table\n", " - Variable 'TREFHT' being added to table\n", " - Variable 'TS' being added to table\n", " - Variable 'U' being added to table\n", " Variable 'U' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'U10' being added to table\n", " - Variable 'ICEFRAC' being added to table\n", " - Variable 'RESTOM' being added to table\n", " - Variable 'LANDFRAC' being added to table\n", " - Variable 'OCNFRAC' being added to table\n", " - Variable 'CLDHGH' being added to table\n", " - Variable 'CLDICE' being added to table\n", " Variable 'CLDICE' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'CLDLIQ' being added to table\n", " Variable 'CLDLIQ' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'CLDLOW' being added to table\n", " - Variable 'CLDMED' being added to table\n", " - Variable 'CLDTOT' being added to table\n", " - Variable 'CLOUD' being added to table\n", " Variable 'CLOUD' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'FLNS' being added to table\n", " - Variable 'FLNT' being added to table\n", " - Variable 'FLNTC' being added to table\n", " - Variable 'FSNS' being added to table\n", " - Variable 'FSNT' being added to table\n", " - Variable 'FSNTC' being added to table\n", " - Variable 'LHFLX' being added to table\n", " - Variable 'LWCF' being added to table\n", " - Variable 'OMEGA500' being added to table\n", " - Variable 'PBLH' being added to table\n", " - Variable 'PRECL' being added to table\n", " - Variable 'PRECT' being added to table\n", " - Variable 'PRECSL' being added to table\n", " - Variable 'PRECSC' being added to table\n", " - Variable 'PRECSC' being added to table\n", " - Variable 'PRECC' being added to table\n", " - Variable 'PS' being added to table\n", " - Variable 'PSL' being added to table\n", " - Variable 'QFLX' being added to table\n", " - Variable 'Q' being added to table\n", " Variable 'Q' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'RELHUM' being added to table\n", " Variable 'RELHUM' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'SHFLX' being added to table\n", " - Variable 'SST' being added to table\n", " - Variable 'SWCF' being added to table\n", " - Variable 'T' being added to table\n", " Variable 'T' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'TAUX' being added to table\n", " - Variable 'TAUY' being added to table\n", " - Variable 'TGCLDIWP' being added to table\n", " - Variable 'TGCLDLWP' being added to table\n", " - Variable 'TMQ' being added to table\n", " - Variable 'TREFHT' being added to table\n", " - Variable 'TS' being added to table\n", " - Variable 'U' being added to table\n", " Variable 'U' has a vertical dimension, which is currently not supported for the AMWG Table. Skipping...\n", " - Variable 'U10' being added to table\n", " - Variable 'ICEFRAC' being added to table\n", " - Variable 'RESTOM' being added to table\n", " ...AMWG variable table has been generated successfully.\n", "\n", " Making comparison table...\n", " ... Comparison table has been generated successfully" ] }, { "cell_type": "code", "execution_count": null, "id": "b474d499-3716-479a-8df4-507bc78ef0a0", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "7e7d1baa-1228-478e-9b38-e146380b7e04", "metadata": {}, "source": [ "##### Run Plotting Scripts\n", "\n", "Create plots.\n", "This call uses the `plotting_scripts` specified in the run-time config file\n", "\n", "```{admonition} Attention!\n", ":class: warning\n", "This will take some time so please be patient!\n", "```" ] }, { "cell_type": "code", "execution_count": null, "id": "606af085-abda-4361-a1df-aba8fa623d66", "metadata": {}, "outputs": [], "source": [ "%%time\n", "adf.create_plots()" ] }, { "cell_type": "markdown", "id": "51387337-d7fc-4bff-80b3-0a278537f3bd", "metadata": {}, "source": [ "Output:" ] }, { "cell_type": "markdown", "id": "a6f1e0a2-193f-44a6-940e-628cbd7b6a7c", "metadata": {}, "source": [ " Generating lat/lon maps...\n", " NOTE: Plot type is set to png\n", " NOTE: redo_plot is set to False\n", " - lat/lon maps for CLDHGH\n", " - lat/lon maps for CLDICE\n", " - lat/lon maps for CLDLIQ\n", " - lat/lon maps for CLDLOW\n", " - lat/lon maps for CLDMED\n", " - lat/lon maps for CLDTOT\n", " - lat/lon maps for CLOUD\n", " - lat/lon maps for FLNS\n", " - lat/lon maps for FLNT\n", " - lat/lon maps for FLNTC\n", " - lat/lon maps for FSNS\n", " - lat/lon maps for FSNT\n", " - lat/lon maps for FSNTC\n", " - lat/lon maps for LHFLX\n", " - lat/lon maps for LWCF\n", " - lat/lon maps for OMEGA500\n", " - lat/lon maps for PBLH\n", " - lat/lon maps for PRECL\n", " - lat/lon maps for PRECT\n", " - lat/lon maps for PRECSL\n", " - lat/lon maps for PRECSC\n", " - lat/lon maps for PRECSC\n", " - lat/lon maps for PRECC\n", " - lat/lon maps for PS\n", " - lat/lon maps for PSL\n", " - lat/lon maps for QFLX\n", " - lat/lon maps for Q\n", " - lat/lon maps for RELHUM\n", " - lat/lon maps for SHFLX\n", " - lat/lon maps for SST\n", " - lat/lon maps for SWCF\n", " - lat/lon maps for T\n", " - lat/lon maps for TAUX\n", " - lat/lon maps for TAUY\n", " - lat/lon maps for TGCLDIWP\n", " - lat/lon maps for TGCLDLWP\n", " - lat/lon maps for TMQ\n", " - lat/lon maps for TREFHT\n", " - lat/lon maps for TS\n", " - lat/lon maps for U\n", " - lat/lon maps for U10\n", " - lat/lon maps for ICEFRAC\n", " - lat/lon maps for OCNFRAC\n", " - lat/lon maps for LANDFRAC\n", " ...lat/lon maps have been generated successfully.\n", "\n", " Generating lat/lon vector maps...\n", " NOTE: redo_plot is set to False\n", " - lat/lon vector maps for TAUX,TAUY\n", " Some vectors at source domain corners may not have been transformed correctly\n", " Some vectors at source domain corners may not have been transformed correctly\n", " - lat/lon vector maps for U,V\n", " ERROR: Did not find any oclim_fils. Will try to skip.\n", " INFO: Data Location, dclimo_loc is /glade/scratch/richling/ADF/adf_tutorial/data/regrid\n", " INFO: The glob is: f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_V_*.nc\n", " ...lat/lon vector maps have been generated successfully.\n", "\n", " Generating zonal mean plots...\n", " NOTE: Plot type is set to png\n", " NOTE: redo_plot is set to False\n", " - zonal mean plots for CLDHGH\n", " - zonal mean plots for CLDICE\n", " CLDICE has lev dimension.\n", " - zonal mean plots for CLDLIQ\n", " CLDLIQ has lev dimension.\n", " - zonal mean plots for CLDLOW\n", " - zonal mean plots for CLDMED\n", " - zonal mean plots for CLDTOT\n", " - zonal mean plots for CLOUD\n", " CLOUD has lev dimension.\n", " - zonal mean plots for FLNS\n", " - zonal mean plots for FLNT\n", " - zonal mean plots for FLNTC\n", " - zonal mean plots for FSNS\n", " - zonal mean plots for FSNT\n", " - zonal mean plots for FSNTC\n", " - zonal mean plots for LHFLX\n", " - zonal mean plots for LWCF\n", " - zonal mean plots for OMEGA500\n", " - zonal mean plots for PBLH\n", " - zonal mean plots for PRECL\n", " - zonal mean plots for PRECT\n", " - zonal mean plots for PRECSL\n", " - zonal mean plots for PRECSC\n", " - zonal mean plots for PRECSC\n", " - zonal mean plots for PRECC\n", " - zonal mean plots for PS\n", " - zonal mean plots for PSL\n", " - zonal mean plots for QFLX\n", " - zonal mean plots for Q\n", " Q has lev dimension.\n", " - zonal mean plots for RELHUM\n", " RELHUM has lev dimension.\n", " - zonal mean plots for SHFLX\n", " - zonal mean plots for SST\n", " - zonal mean plots for SWCF\n", " - zonal mean plots for T\n", " T has lev dimension.\n", " - zonal mean plots for TAUX\n", " - zonal mean plots for TAUY\n", " - zonal mean plots for TGCLDIWP\n", " - zonal mean plots for TGCLDLWP\n", " - zonal mean plots for TMQ\n", " - zonal mean plots for TREFHT\n", " - zonal mean plots for TS\n", " - zonal mean plots for U\n", " U has lev dimension.\n", " - zonal mean plots for U10\n", " - zonal mean plots for ICEFRAC\n", " - zonal mean plots for OCNFRAC\n", " - zonal mean plots for LANDFRAC\n", " ...Zonal mean plots have been generated successfully.\n", "\n", " Generating meridional mean plots...\n", " NOTE: Plot type is set to png\n", " NOTE: redo_plot is set to False\n", " - meridional mean plots for CLDHGH\n", " - meridional mean plots for CLDICE\n", " CLDICE has lev dimension.\n", " - meridional mean plots for CLDLIQ\n", " CLDLIQ has lev dimension.\n", " - meridional mean plots for CLDLOW\n", " - meridional mean plots for CLDMED\n", " - meridional mean plots for CLDTOT\n", " - meridional mean plots for CLOUD\n", " CLOUD has lev dimension.\n", " - meridional mean plots for FLNS\n", " - meridional mean plots for FLNT\n", " - meridional mean plots for FLNTC\n", " - meridional mean plots for FSNS\n", " - meridional mean plots for FSNT\n", " - meridional mean plots for FSNTC\n", " - meridional mean plots for LHFLX\n", " - meridional mean plots for LWCF\n", " - meridional mean plots for OMEGA500\n", " - meridional mean plots for PBLH\n", " - meridional mean plots for PRECL\n", " - meridional mean plots for PRECT\n", " - meridional mean plots for PRECSL\n", " - meridional mean plots for PRECSC\n", " - meridional mean plots for PRECSC\n", " - meridional mean plots for PRECC\n", " - meridional mean plots for PS\n", " - meridional mean plots for PSL\n", " - meridional mean plots for QFLX\n", " - meridional mean plots for Q\n", " Q has lev dimension.\n", " - meridional mean plots for RELHUM\n", " RELHUM has lev dimension.\n", " - meridional mean plots for SHFLX\n", " - meridional mean plots for SST\n", " - meridional mean plots for SWCF\n", " - meridional mean plots for T\n", " T has lev dimension.\n", " - meridional mean plots for TAUX\n", " - meridional mean plots for TAUY\n", " - meridional mean plots for TGCLDIWP\n", " - meridional mean plots for TGCLDLWP\n", " - meridional mean plots for TMQ\n", " - meridional mean plots for TREFHT\n", " - meridional mean plots for TS\n", " - meridional mean plots for U\n", " U has lev dimension.\n", " - meridional mean plots for U10\n", " - meridional mean plots for ICEFRAC\n", " - meridional mean plots for OCNFRAC\n", " - meridional mean plots for LANDFRAC\n", " ...Meridional mean plots have been generated successfully.\n", "\n", " Generating polar maps...\n", " NOTE: Plot type is set to png\n", " NOTE: redo_plot is set to False\n", " - polar maps for CLDHGH\n", " - polar maps for CLDICE\n", " - polar maps for CLDLIQ\n", " - polar maps for CLDLOW\n", " - polar maps for CLDMED\n", " - polar maps for CLDTOT\n", " - polar maps for CLOUD\n", " - polar maps for FLNS\n", " - polar maps for FLNT\n", " - polar maps for FLNTC\n", " - polar maps for FSNS\n", " - polar maps for FSNT\n", " - polar maps for FSNTC\n", " - polar maps for LHFLX\n", " - polar maps for LWCF\n", " - polar maps for OMEGA500\n", " - polar maps for PBLH\n", " - polar maps for PRECL\n", " - polar maps for PRECT\n", " - polar maps for PRECSL\n", " - polar maps for PRECSC\n", " - polar maps for PRECSC\n", " - polar maps for PRECC\n", " - polar maps for PS\n", " - polar maps for PSL\n", " - polar maps for QFLX\n", " - polar maps for Q\n", " - polar maps for RELHUM\n", " - polar maps for SHFLX\n", " - polar maps for SST\n", " - polar maps for SWCF\n", " - polar maps for T\n", " - polar maps for TAUX\n", " - polar maps for TAUY\n", " - polar maps for TGCLDIWP\n", " - polar maps for TGCLDLWP\n", " - polar maps for TMQ\n", " - polar maps for TREFHT\n", " - polar maps for TS\n", " - polar maps for U\n", " - polar maps for U10\n", " - polar maps for ICEFRAC\n", " - polar maps for OCNFRAC\n", " - polar maps for LANDFRAC\n", " ...polar maps have been generated successfully.\n", "\n", " Generating Taylor Diagrams...\n", " Ambiguous plotting location since all cases go on same plot. Will put them in first location: /glade/scratch/richling/ADF/adf_tutorial/plots/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001_1995_2000_vs_f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_1995_2000\n", " NOTE: Plot type is set to png\n", " NOTE: redo_plot is set to False\n", " - Plotting Taylor Diagram, ANN\n", " Taylor Diagram: completed ANN. \n", " File: /glade/scratch/richling/ADF/adf_tutorial/plots/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001_1995_2000_vs_f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_1995_2000/TaylorDiag_ANN_Special_Mean.png\n", " - Plotting Taylor Diagram, DJF\n", " Taylor Diagram: completed DJF. \n", " File: /glade/scratch/richling/ADF/adf_tutorial/plots/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001_1995_2000_vs_f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_1995_2000/TaylorDiag_DJF_Special_Mean.png\n", " - Plotting Taylor Diagram, JJA\n", " Taylor Diagram: completed JJA. \n", " File: /glade/scratch/richling/ADF/adf_tutorial/plots/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001_1995_2000_vs_f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_1995_2000/TaylorDiag_JJA_Special_Mean.png\n", " - Plotting Taylor Diagram, MAM\n", " Taylor Diagram: completed MAM. \n", " File: /glade/scratch/richling/ADF/adf_tutorial/plots/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001_1995_2000_vs_f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_1995_2000/TaylorDiag_MAM_Special_Mean.png\n", " - Plotting Taylor Diagram, SON\n", " Taylor Diagram: completed SON. \n", " File: /glade/scratch/richling/ADF/adf_tutorial/plots/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001_1995_2000_vs_f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_1995_2000/TaylorDiag_SON_Special_Mean.png\n", " ...Taylor Diagrams have been generated successfully.\n", "\n", " Generating qbo plots...\n", " NOTE: redo_plot is set to False\n", " QBO plots will be saved here: /glade/scratch/richling/ADF/adf_tutorial/plots/f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001_1995_2000_vs_f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001_1995_2000\n", " ...QBO plots have been generated successfully.\n", " CPU times: user 30min 24s, sys: 30.9 s, total: 30min 55s\n", " Wall time: 33min 22s" ] }, { "cell_type": "markdown", "id": "5b78d753-fa01-4b9d-ad26-ae309ea06bdf", "metadata": {}, "source": [ "

Run Website Generation Scripts

\n", "\n", "Create website if requested in the run-time config file" ] }, { "cell_type": "code", "execution_count": null, "id": "06e4552c-6379-4943-959a-bb2b2e362955", "metadata": {}, "outputs": [], "source": [ "if adf.create_html:\n", " adf.create_website()" ] }, { "cell_type": "markdown", "id": "bdf9f8d1-dac8-4249-822b-6d91f1229e16", "metadata": {}, "source": [ "

Output:

" ] }, { "cell_type": "markdown", "id": "b9e3adc6-eb24-4de0-bda8-1464e090a191", "metadata": {}, "source": [ " Generating Diagnostics webpages...\n", " ...Webpages have been generated successfully." ] }, { "cell_type": "markdown", "id": "447a17a3-89af-4fb5-8b59-615562fd5e82", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "id": "789a3fe1-b94d-48ec-80bf-da543c2742d3", "metadata": {}, "source": [ "### Step 2: Analyze ADF Output\n", "\n", "Once the ADF has completed (either here in Jupyter or previously from terminal run) we can use the `adf` object to gain access to everything available in the ADF. This could be very useful if you want to develop diagsnostics separate from the ADF in a Jupyter Notebook" ] }, { "cell_type": "raw", "id": "77fc2bde-f29a-4a56-be77-4f59fcf7a8bb", "metadata": {}, "source": [ " config_cam_baseline_example.yaml" ] }, { "cell_type": "markdown", "id": "73a7385a-8a41-4b2d-bdd2-cef633109509", "metadata": {}, "source": [ "The `adf` object can be searched for any quantity of interest. Using the dot, `.` and tab function in Jupyter, we can get a list of all the available instances, classes, functions, etc:\n", "\n", "![adf objects](../../images/adf_dir.png)\n", "\n", "Or calling the dir command on the ADF object will print a list of all available objects:" ] }, { "cell_type": "code", "execution_count": 24, "id": "e6136e1a-3bac-4093-a249-aef4fcf568d0", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['_AdfBase__debug_log',\n", " '_AdfConfig__config_dict',\n", " '_AdfConfig__create_search_dict',\n", " '_AdfConfig__expand_yaml_var_ref',\n", " '_AdfConfig__kword_pattern',\n", " '_AdfConfig__search_dict',\n", " '_AdfDiag__analysis_scripts',\n", " '_AdfDiag__cvdp_info',\n", " '_AdfDiag__diag_scripts_caller',\n", " '_AdfDiag__function_caller',\n", " '_AdfDiag__plotting_scripts',\n", " '_AdfDiag__regridding_scripts',\n", " '_AdfDiag__time_averaging_scripts',\n", " '_AdfInfo__base_nickname',\n", " '_AdfInfo__basic_info',\n", " '_AdfInfo__cam_bl_climo_info',\n", " '_AdfInfo__cam_climo_info',\n", " '_AdfInfo__compare_obs',\n", " '_AdfInfo__diag_var_list',\n", " '_AdfInfo__eyear_baseline',\n", " '_AdfInfo__eyears',\n", " '_AdfInfo__num_cases',\n", " '_AdfInfo__num_procs',\n", " '_AdfInfo__plot_location',\n", " '_AdfInfo__syear_baseline',\n", " '_AdfInfo__syears',\n", " '_AdfInfo__test_nicknames',\n", " '_AdfObs__var_obs_dict',\n", " '_AdfObs__variable_defaults',\n", " '_AdfWeb__case_web_paths',\n", " '_AdfWeb__plot_type_multi',\n", " '_AdfWeb__plot_type_order',\n", " '_AdfWeb__website_data',\n", " '__class__',\n", " '__delattr__',\n", " '__dict__',\n", " '__dir__',\n", " '__doc__',\n", " '__eq__',\n", " '__format__',\n", " '__ge__',\n", " '__getattribute__',\n", " '__gt__',\n", " '__hash__',\n", " '__init__',\n", " '__init_subclass__',\n", " '__le__',\n", " '__lt__',\n", " '__module__',\n", " '__ne__',\n", " '__new__',\n", " '__reduce__',\n", " '__reduce_ex__',\n", " '__repr__',\n", " '__setattr__',\n", " '__sizeof__',\n", " '__str__',\n", " '__subclasshook__',\n", " '__weakref__',\n", " 'add_diag_var',\n", " 'add_website_data',\n", " 'baseline_climo_dict',\n", " 'basic_info_dict',\n", " 'cam_climo_dict',\n", " 'case_nicknames',\n", " 'climo_yrs',\n", " 'compare_obs',\n", " 'create_climo',\n", " 'create_html',\n", " 'create_plots',\n", " 'create_time_series',\n", " 'create_website',\n", " 'debug_log',\n", " 'diag_var_list',\n", " 'end_diag_fail',\n", " 'expand_references',\n", " 'get_baseline_info',\n", " 'get_basic_info',\n", " 'get_cam_info',\n", " 'get_cvdp_info',\n", " 'num_cases',\n", " 'num_procs',\n", " 'perform_analyses',\n", " 'plot_location',\n", " 'plotting_scripts',\n", " 'read_config_var',\n", " 'regrid_climo',\n", " 'setup_run_cvdp',\n", " 'var_obs_dict',\n", " 'variable_defaults']" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dir(adf)" ] }, { "cell_type": "markdown", "id": "37c73012-17e5-4468-8fb9-76e474321760", "metadata": {}, "source": [ "

Grab the case name instances:

" ] }, { "cell_type": "code", "execution_count": 10, "id": "4b1c3a39-ae17-4ef3-ae8a-263c43f1a3ac", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['f.cam6_3_106.FLTHIST_v0a.ne30.dcs_effgw_rdg.001']\n", "f.cam6_3_106.FLTHIST_v0a.ne30.dcs_non-ogw.001\n" ] } ], "source": [ "case_names = adf.get_cam_info(\"cam_case_name\",required=True)\n", "print(case_names)\n", "base_name = adf.get_baseline_info(\"cam_case_name\",required=True)\n", "print(base_name)" ] }, { "cell_type": "markdown", "id": "2785ad59-a456-4984-be79-260940d9ff11", "metadata": {}, "source": [ "

Grab the case climo years instances:

" ] }, { "cell_type": "code", "execution_count": 11, "id": "fc745257-d75c-45e6-9843-ca1c727324b6", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'syears': [1995],\n", " 'eyears': [2000],\n", " 'syear_baseline': 1995,\n", " 'eyear_baseline': 2000}" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adf.climo_yrs" ] }, { "cell_type": "markdown", "id": "da9e7dee-2d67-420d-883b-1fb9337932f3", "metadata": {}, "source": [ "

Grab the listed CAM variables from the config file:

" ] }, { "cell_type": "code", "execution_count": 13, "id": "4f4c4db8-f481-4a11-b0e3-1799d98f2864", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['CLDHGH',\n", " 'CLDICE',\n", " 'CLDLIQ',\n", " 'CLDLOW',\n", " 'CLDMED',\n", " 'CLDTOT',\n", " 'CLOUD',\n", " 'FLNS',\n", " 'FLNT',\n", " 'FLNTC',\n", " 'FSNS',\n", " 'FSNT',\n", " 'FSNTC',\n", " 'LHFLX',\n", " 'LWCF',\n", " 'OMEGA500',\n", " 'PBLH',\n", " 'PRECL',\n", " 'PRECT',\n", " 'PRECSL',\n", " 'PRECSC',\n", " 'PRECSC',\n", " 'PRECC',\n", " 'PS',\n", " 'PSL',\n", " 'QFLX',\n", " 'Q',\n", " 'RELHUM',\n", " 'SHFLX',\n", " 'SST',\n", " 'SWCF',\n", " 'T',\n", " 'TAUX',\n", " 'TAUY',\n", " 'TGCLDIWP',\n", " 'TGCLDLWP',\n", " 'TMQ',\n", " 'TREFHT',\n", " 'TS',\n", " 'U',\n", " 'U10',\n", " 'ICEFRAC',\n", " 'OCNFRAC',\n", " 'LANDFRAC']" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "var_list = adf.diag_var_list\n", "list(var_list)" ] }, { "cell_type": "markdown", "id": "5420eb2d-0e3d-4f44-94b9-2622bb5912dd", "metadata": {}, "source": [ "

Let's build a custom diagnsotics plot!

" ] }, { "cell_type": "markdown", "id": "d1c18fd8-fc23-4991-ad42-b7c006c54697", "metadata": {}, "source": [ "#### Stratocumulus Over California Example\n", "\n", "Now that we have the ADF done and still have access to all the information contained in the ADF, let's run a custom routine to calculate and plot stratocumulus clouds over California. \n", "\n", "The point of this example is to highlight using a script not contained in the ADF to simulate custom analysis (that may be a one off diagnsotics need), all within the same notebook as our ADF run.\n", "\n", "Again, this can be very helpful becasue the ADF object has all the paths for the input/output data and case information already saved for convienence." ] }, { "cell_type": "code", "execution_count": 18, "id": "cf5e166f-70a3-496d-bd84-1e5bd873ac04", "metadata": {}, "outputs": [], "source": [ "# Get the location of the climo files\n", "case_climo_loc = adf.get_cam_info('cam_climo_loc', required=True)\n", "data_loc = adf.get_baseline_info(\"cam_climo_loc\", required=True)" ] }, { "cell_type": "code", "execution_count": 12, "id": "4309a554-0deb-4bd8-b222-d28e56d81892", "metadata": {}, "outputs": [], "source": [ "import matplotlib as mpl\n", "# change color for each case\n", "case_colors = [mpl.cm.tab20(i) for i, case in enumerate(case_names)]" ] }, { "cell_type": "code", "execution_count": 25, "id": "62dbee3d-80e8-4382-a689-b28a3a97bc3d", "metadata": {}, "outputs": [], "source": [ "import geocat.comp as gc # use geocat's interpolation\n", "\n", "def process_case(climo_loc, latitude, longitude, pressurelevels):\n", " fils = sorted(list(Path(climo_loc).glob(f\"*_T_*.nc\")))\n", " temperature = xr.open_mfdataset(fils)['T'].sel(lat=latitude, lon=longitude).compute()\n", " fils = sorted(list(Path(climo_loc).glob(f\"*_Q_*.nc\")))\n", " vapor = xr.open_mfdataset(fils)['Q'].sel(lat=latitude, lon=longitude).compute()\n", " fils = sorted(list(Path(climo_loc).glob(f\"*_CLDLIQ_*.nc\")))\n", " liquid = xr.open_mfdataset(fils)['CLDLIQ'].sel(lat=latitude, lon=longitude).compute()\n", " # In one of these, we also need the hybrid-sigma coefficients\n", " fils = sorted(list(Path(climo_loc).glob(f\"*_PS_*.nc\")))\n", " ps_ds = xr.open_mfdataset(fils)\n", " ps = ps_ds['PS'].sel(lat=latitude, lon=longitude).compute()\n", " hyam = ps_ds['hyam'].isel(time=0).compute() # drop redundant time dimension\n", " hybm = ps_ds['hybm'].isel(time=0).compute()\n", " ps.name = \"PS\"\n", " \n", " # we aren't done. Interpolate to pressure levels here:\n", " t_plev = gc.interp_hybrid_to_pressure(temperature, ps, hyam, hybm, new_levels=pressurelevels, lev_dim='lev').compute()\n", " q_plev = gc.interp_hybrid_to_pressure(vapor, ps, hyam, hybm, new_levels=pressurelevels, lev_dim='lev').compute()\n", " liq_plev = gc.interp_hybrid_to_pressure(liquid, ps, hyam, hybm, new_levels=pressurelevels, lev_dim='lev').compute()\n", " \n", " t_plev.name = \"T\"\n", " q_plev.name = \"Q\"\n", " liq_plev.name = 'CLDLIQ'\n", " \n", " # But hold on, we actually want theta, not T:\n", " p = xr.DataArray(pressurelevels, dims='plev', coords={'plev': t_plev.plev})\n", " # pressurelevels expected in Pa!\n", " theta_plev = t_plev * ((100000. / p)**0.2854) # https://glossary.ametsoc.org/wiki/Potential_temperature & https://glossary.ametsoc.org/wiki/Poisson_constant\n", " \n", " # Still not done -- average over the area:\n", " w = np.cos(np.radians(temperature.lat)) # area weighting\n", " theta_aave = theta_plev.weighted(w).mean(dim=(\"lat\",\"lon\")) # (12months x pressurelevels.shape[0])\n", " theta_aave.name = \"THETA\"\n", " q_aave = q_plev.weighted(w).mean(dim=(\"lat\",\"lon\"))\n", " liq_aave = liq_plev.weighted(w).mean(dim=(\"lat\",\"lon\"))\n", " ps_aave = ps.weighted(w).mean(dim=(\"lat\",\"lon\"))\n", " return xr.merge([theta_aave, q_aave, liq_aave, ps_aave])" ] }, { "cell_type": "code", "execution_count": 14, "id": "3abcd08f-d0bf-47bc-96f6-ac05bce31a2c", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "from cycler import cycler\n", "\n", "def make_plot(ds1, ds2, caselabels):\n", " fig, ax = plt.subplots(figsize=(9,4), ncols=3, sharey=True, constrained_layout=True)\n", " custom_cycler = (cycler(color=['k', 'r', 'y', 'y']) +\n", " cycler(lw=[2, 2, 1, 1]))\n", " [a.set_prop_cycle(custom_cycler) for a in ax]\n", " if ds1['Q'].max() < 0.1:\n", " qscale = 1000. # kg/kg -> mg/kg\n", " else:\n", " qscale = 1\n", " if ds1['CLDLIQ'].max() < 0.001:\n", " lscale = 1000.*1000. # kg/kg -> ug/kg\n", " else:\n", " lscale = 1\n", "\n", " ax[0].plot(ds1['Q']*qscale, ds1['plev']/100, label=caselabels[0])\n", " ax[0].plot(ds2['Q']*qscale, ds2['plev']/100, label=caselabels[1])\n", " ax[0].set_xlim([0,25])\n", " ax[0].set_xlabel(\"Specific Humidity [g/kg]\")\n", " ax[1].plot(ds1['THETA'], ds1['plev']/100)\n", " ax[1].plot(ds2['THETA'], ds2['plev']/100)\n", " ax[1].set_xlabel(\"Potential Temperature [K]\")\n", " ax[1].set_xlim([275, 325])\n", " ax[2].plot(ds1['CLDLIQ']*lscale, ds1['plev']/100)\n", " ax[2].plot(ds2['CLDLIQ']*lscale, ds2['plev']/100)\n", " ax[2].set_xlabel(\"Cloud Liquid [$\\mu$g/kg]\")\n", " ax[2].set_xlim([0,150])\n", " ax[2].invert_yaxis()\n", " ax[0].set_ylabel(\"Pressure\")\n", " [a.spines['top'].set_visible(False) for a in ax]\n", " [a.spines['right'].set_visible(False) for a in ax]\n", " fig.legend(loc='upper left', bbox_to_anchor=(0.0, -0.01))\n", " return fig, ax" ] }, { "cell_type": "code", "execution_count": 16, "id": "82e4b9d1-4b58-4e51-861c-495366abcde2", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "import xarray as xr\n", "import numpy as np\n", "\n", "# define domain\n", "latslice = slice(20,30)\n", "lonslice = slice(230,240) # Klein&Hartmann 1993, Table 1\n", "levels = 100.*np.arange(600.0, 1015., 15) # chosen for convenience. Go finer if native grid is finer." ] }, { "cell_type": "code", "execution_count": 19, "id": "8ee3b454-b11c-4882-a8f4-eb6e15e18820", "metadata": {}, "outputs": [ { "ename": "KeyError", "evalue": "'hyam'", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", "File \u001b[0;32m/glade/u/apps/opt/conda/envs/npl-2023a/lib/python3.9/site-packages/xarray/core/dataset.py:1348\u001b[0m, in \u001b[0;36mDataset._construct_dataarray\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m 1347\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m-> 1348\u001b[0m variable \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_variables\u001b[49m\u001b[43m[\u001b[49m\u001b[43mname\u001b[49m\u001b[43m]\u001b[49m\n\u001b[1;32m 1349\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m:\n", "\u001b[0;31mKeyError\u001b[0m: 'hyam'", "\nDuring handling of the above exception, another exception occurred:\n", "\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[19], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m ref_ds \u001b[38;5;241m=\u001b[39m \u001b[43mprocess_case\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdata_loc\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlatslice\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlonslice\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlevels\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m i, c \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28menumerate\u001b[39m(case_names):\n\u001b[1;32m 4\u001b[0m case_ds \u001b[38;5;241m=\u001b[39m process_case(case_climo_loc[i], latslice, lonslice, levels)\n", "Cell \u001b[0;32mIn[13], line 14\u001b[0m, in \u001b[0;36mprocess_case\u001b[0;34m(climo_loc, latitude, longitude, pressurelevels)\u001b[0m\n\u001b[1;32m 12\u001b[0m ps_ds \u001b[38;5;241m=\u001b[39m xr\u001b[38;5;241m.\u001b[39mopen_mfdataset(fils)\n\u001b[1;32m 13\u001b[0m ps \u001b[38;5;241m=\u001b[39m ps_ds[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPS\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39msel(lat\u001b[38;5;241m=\u001b[39mlatitude, lon\u001b[38;5;241m=\u001b[39mlongitude)\u001b[38;5;241m.\u001b[39mcompute()\n\u001b[0;32m---> 14\u001b[0m hyam \u001b[38;5;241m=\u001b[39m \u001b[43mps_ds\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mhyam\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[38;5;241m.\u001b[39misel(time\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0\u001b[39m)\u001b[38;5;241m.\u001b[39mcompute() \u001b[38;5;66;03m# drop redundant time dimension\u001b[39;00m\n\u001b[1;32m 15\u001b[0m hybm \u001b[38;5;241m=\u001b[39m ps_ds[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mhybm\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39misel(time\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0\u001b[39m)\u001b[38;5;241m.\u001b[39mcompute()\n\u001b[1;32m 16\u001b[0m ps\u001b[38;5;241m.\u001b[39mname \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mPS\u001b[39m\u001b[38;5;124m\"\u001b[39m\n", "File \u001b[0;32m/glade/u/apps/opt/conda/envs/npl-2023a/lib/python3.9/site-packages/xarray/core/dataset.py:1439\u001b[0m, in \u001b[0;36mDataset.__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m 1437\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39misel(\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkey)\n\u001b[1;32m 1438\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m utils\u001b[38;5;241m.\u001b[39mhashable(key):\n\u001b[0;32m-> 1439\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_construct_dataarray\u001b[49m\u001b[43m(\u001b[49m\u001b[43mkey\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1440\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m utils\u001b[38;5;241m.\u001b[39miterable_of_hashable(key):\n\u001b[1;32m 1441\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_copy_listed(key)\n", "File \u001b[0;32m/glade/u/apps/opt/conda/envs/npl-2023a/lib/python3.9/site-packages/xarray/core/dataset.py:1350\u001b[0m, in \u001b[0;36mDataset._construct_dataarray\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m 1348\u001b[0m variable \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_variables[name]\n\u001b[1;32m 1349\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m:\n\u001b[0;32m-> 1350\u001b[0m _, name, variable \u001b[38;5;241m=\u001b[39m \u001b[43m_get_virtual_variable\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_variables\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdims\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1352\u001b[0m needed_dims \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mset\u001b[39m(variable\u001b[38;5;241m.\u001b[39mdims)\n\u001b[1;32m 1354\u001b[0m coords: \u001b[38;5;28mdict\u001b[39m[Hashable, Variable] \u001b[38;5;241m=\u001b[39m {}\n", "File \u001b[0;32m/glade/u/apps/opt/conda/envs/npl-2023a/lib/python3.9/site-packages/xarray/core/dataset.py:186\u001b[0m, in \u001b[0;36m_get_virtual_variable\u001b[0;34m(variables, key, dim_sizes)\u001b[0m\n\u001b[1;32m 184\u001b[0m split_key \u001b[38;5;241m=\u001b[39m key\u001b[38;5;241m.\u001b[39msplit(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m.\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;241m1\u001b[39m)\n\u001b[1;32m 185\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(split_key) \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m2\u001b[39m:\n\u001b[0;32m--> 186\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m(key)\n\u001b[1;32m 188\u001b[0m ref_name, var_name \u001b[38;5;241m=\u001b[39m split_key\n\u001b[1;32m 189\u001b[0m ref_var \u001b[38;5;241m=\u001b[39m variables[ref_name]\n", "\u001b[0;31mKeyError\u001b[0m: 'hyam'" ] } ], "source": [ "ref_ds = process_case(data_loc, latslice, lonslice, levels)\n", "\n", "for i, c in enumerate(case_names):\n", " case_ds = process_case(case_climo_loc[i], latslice, lonslice, levels)\n", " print(\"case ready\")\n", " for s in seasons:\n", " ref_season = ref_ds.sel(time=seasons[s]).mean(dim='time')\n", " case_season = case_ds.sel(time=seasons[s]).mean(dim='time')\n", " # ** Ready to make plot **\n", " labels = [data_name, c]\n", " casefig, caseax = make_plot(ref_season, case_season, labels)\n", " casefig.suptitle(f\"California Stratocumulus, {s}\")\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": null, "id": "f9f25f23-3f8e-494b-b44b-523737516e75", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "42e59fc3-96f9-490b-bd58-7515aacfa559", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "1b4c896f-2615-4e27-a9eb-994b2504d6e7", "metadata": {}, "source": [] }, { "cell_type": "markdown", "id": "0927a805-1aea-4cbf-9351-e686bd3d5f1b", "metadata": {}, "source": [] }, { "cell_type": "markdown", "id": "e46eee54-c674-4e65-b545-4f42959befae", "metadata": {}, "source": [] } ], "metadata": { "kernelspec": { "display_name": "NPL 2023a", "language": "python", "name": "npl-2023a" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 5 }