Example: convert E3SMv2 SCM output to DEPHY format

Example: convert E3SMv2 SCM output to DEPHY format#

Code to read E3SMv2 output files and write to DEPHY format (NetCDF)

Contributed by Ann Fridlind from NASA/GISS, Meng Zhang from LLNL

Import libraries#

import xarray as xr
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import csv
import os
import netCDF4
import datetime as dt
from netCDF4 import Dataset
import scipy.stats
from scipy import interpolate

Specify directory locations#

If on the ARM JupyterHub, it is recommended to create and specify a local directory that is outside of the COMBLE-MIP repository to upload raw model output files in your model’s format.

Processed domain-mean outputs are invited for commit to the GitHub repository on a user-specified branch under /comble-mip/output_scm/YOUR_MODEL_NAME/sandbox/YOUR_OUTPUT_FILE_NAME. These can be committed and removed at any time.

If you are able to make a run without ice, it is requested to append ‘noice’ to YOUR_OUTPUT_FILE_NAME, so that it can readily be automatically compared with the baseline and other liquid-only runs.

# specify input and output file names: versions of ModelE3 with and without ice

# Phys
# my_input_suffix = '.eam.h0.2020-03-12-79200.nc'
# my_output_suffix = 'Phys_FixN_def_z0_alt_no_ugvg.nc'

# Phys without ice
my_input_suffix = '.eam.h0.2020-03-12-79200.nc'
my_output_suffix = 'Phys_FixN_noice_def_z0_alt_no_ugvg.nc'

# Tun1
# my_input_suffix = 't698ml.nc'
# my_output_suffix = 'Tun1_FixN.nc'

# Tun1 without ice
# my_input_suffix = 't698ml_noice.nc'
# my_output_suffix = 'Tun1_FixN_noice.nc'

# Tun2
# my_input_suffix = 't705ml.nc'
# my_output_suffix = 'Tun2_FixN.nc'

# Tun2 without ice
# my_input_suffix = 't705ml_noice.nc'
# my_output_suffix = 'Tun2_FixN_noice.nc'

# specify local source directories (with subdirectories for spin up over ice and restart over water)
my_input_dir = '/home/mzhang1/Data_test/COMBLE-MIP-SCM/'

# specify Github scratch directory where processed model output will be committed (automate later)
my_output_filename = 'E3SMv2-' + my_output_suffix
my_gitdir = '/home/mzhang1/Data_test/COMBLE-MIP-SCM/gitdir/' #'../../output_scm/modelE/sandbox/'

Read E3SM data#

Read single file containing all output data#

Note: ERROR 1: PROJ… message can be ignored here.

input_filename = my_input_dir + 'COMBLE-MIP_v2_4_e3smv2_1_FixN_noice' + my_input_suffix
model_data = xr.open_dataset(input_filename)

# check if the run contains ice variables
do_ice = bool(max(model_data['TGCLDIWP'].values)>0.)
print('do_ice = ',do_ice)

# full parameter list
model_data

do_ice =  False

<xarray.Dataset>
Dimensions:           (ncol: 1, lev: 72, ilev: 73, time: 41, nbnd: 2)
Coordinates:
  * lev               (lev) float64 0.1238 0.1828 0.2699 ... 986.2 993.8 998.5
  * ilev              (ilev) float64 0.1 0.1477 0.218 ... 990.5 997.0 1e+03
  * time              (time) datetime64[ns] 2020-03-12T22:00:00 ... 2020-03-1...
Dimensions without coordinates: ncol, nbnd
Data variables: (12/333)
    lat               (ncol) float64 ...
    lon               (ncol) float64 ...
    area              (ncol) float64 ...
    hyam              (lev) float64 ...
    hybm              (lev) float64 ...
    P0                float64 ...
    ...                ...
    m_ozone_c         (time, lev, ncol) float32 ...
    mu_cloud          (time, lev, ncol) float32 ...
    ozone             (time, lev, ncol) float32 ...
    wat_a1            (time, lev, ncol) float32 ...
    wat_a2            (time, lev, ncol) float32 ...
    wat_a3            (time, lev, ncol) float32 ...
Attributes: (12/19)
    ne:                4
    np:                4
    title:             EAM History file information
    source:            E3SM Atmosphere Model
    source_id:         0f362f469f
    product:           model-output
    ...                ...
    institution_id:    E3SM-Project
    institution:       LLNL (Lawrence Livermore National Laboratory, Livermor...
    contact:           e3sm-data-support@llnl.gov
    initial_file:      /global/cfs/cdirs/e3sm/inputdata/atm/cam/inic/homme/ca...
    topography_file:   /global/cfs/cdirs/e3sm/inputdata/atm/cam/topo/USGS-gto...
    time_period_freq:  minute_30
# 2D variables
icol = 0

# assign pressure layer
dum_var = model_data['hyam']*0.
model_data = model_data.assign(layer = dum_var + model_data['lev'].data)

# calculate air pressure
dum_var = model_data['T'][:,:,icol]*0.
dum_pa = model_data['hyam'].data * model_data['P0'].data + model_data['hybm'].data * model_data['PS'].data
model_data = model_data.assign(pa = dum_var + dum_pa)

# calculate layer top pressure
dum_var = model_data['T'][:,:,icol]*0.
dum_pe = model_data['hyai'].data * model_data['P0'].data + model_data['hybi'].data * model_data['PS'].data
model_data = model_data.assign(pe = dum_var + dum_pe[:,:-1])

# assign surface pressure
dum_var = model_data['PS'][:,icol]*0.
model_data = model_data.assign(ps = dum_var + model_data['PS'][:,icol].data)

# assign surface friction velocity
dum_var = model_data['USTAR'][:,icol]*0.
model_data = model_data.assign(ustar = dum_var + model_data['USTAR'][:,icol].data)

# assign surface temperature
dum_var = model_data['TS'][:,icol]*0.
model_data = model_data.assign(ts = dum_var + model_data['TS'][:,icol].data)

# assign surface upward sensible heat flux
dum_var = model_data['SHFLX'][:,icol]*0.
model_data = model_data.assign(hfss = dum_var + model_data['SHFLX'][:,icol].data)

# assign surface upward latent heat flux
dum_var = model_data['LHFLX'][:,icol]*0.
model_data = model_data.assign(hfls = dum_var + model_data['LHFLX'][:,icol].data)

# assign obukhov length
dum_var = model_data['OBKLEN'][:,icol]*0.
model_data = model_data.assign(ol = dum_var + model_data['OBKLEN'][:,icol].data)

# assign PBL height
dum_var = model_data['PBLH'][:,icol]*0.
model_data = model_data.assign(pblh = dum_var + model_data['PBLH'][:,icol].data)

# calculate liquid water path
dum_var = model_data['TGCLDLWP'][:,icol]*0.
pint = model_data['hyai'].data * model_data['P0'].data + model_data['hybi'].data * model_data['PS'].data
pdel = pint[:,1:]-np.roll(pint, 1, axis=1)[:,1:]
dum_lwpc = (model_data['CLDLIQ'][:,:,0].data * pdel / 9.80665).sum(axis=1)
model_data = model_data.assign(lwpc = dum_var + dum_lwpc)

# calculate rain water path
dum_var = model_data['TGCLDLWP'][:,icol]*0.
dum_lwpr = (model_data['RAINQM'][:,:,0].data * pdel / 9.80665).sum(axis=1)
model_data = model_data.assign(lwpr = dum_var + dum_lwpr)

# calculate ice water path
dum_var = model_data['TGCLDLWP'][:,icol]*0.
dum_iwp = (model_data['CLDICE'][:,:,0].data * pdel / 9.80665).sum(axis=1)
dum_iwp = dum_iwp + (model_data['SNOWQM'][:,:,0].data * pdel / 9.80665).sum(axis=1)
model_data = model_data.assign(iwp = dum_var + dum_iwp)

# assign cf
dum_var = model_data['CLDTOT'][:,icol]*0.
model_data = model_data.assign(cf = dum_var + model_data['CLDTOT'][:,icol].data)

# assign cfc
dum_var = model_data['FREQZM'][:,icol]*0.
model_data = model_data.assign(cfc = dum_var + model_data['CONCLD'][:,:,icol].max(axis=1).data)

# # assign optical depth
# dum_var = model_data['TOT_ICLD_VISTAU']*0.
# model_data = model_data.assign(od = dum_var + model_data['TOT_ICLD_VISTAU'].data)

# # assign optical depth of liquid cloud
# dum_var = model_data['LIQ_ICLD_VISTAU']*0.
# model_data = model_data.assign(odlc = dum_var + model_data['LIQ_ICLD_VISTAU'].data)

# calculate pr
dum_var = model_data['PRECL'][:,icol]*0.
model_data = model_data.assign(pr = dum_var + (model_data['PRECC'][:,icol].data + model_data['PRECL'][:,icol].data) * 1000.)

# calculate pri
dum_var = model_data['PRECL'][:,icol]*0.
model_data = model_data.assign(pri = dum_var + (model_data['PRECSC'][:,icol].data + model_data['PRECSL'][:,icol].data) * 1000.)

# calculate rlut
dum_var = model_data['FLUT'][:,icol]*0.
model_data = model_data.assign(rlut = dum_var + model_data['FLUT'][:,icol].data)

# calculate rlds
dum_var = model_data['FLDS'][:,icol]*0.
model_data = model_data.assign(rlds = dum_var + model_data['FLDS'][:,icol].data)

# calculate rlus
dum_var = model_data['FLNS'][:,icol]*0.
model_data = model_data.assign(rlus = dum_var + model_data['FLNS'][:,icol].data + model_data['FLDS'][:,icol].data)

# 3D variables
# assign qlcs
dum_var = model_data['CLDLIQ'][:,:,icol]*0.
model_data = model_data.assign(qlcs = dum_var + model_data['CLDLIQ'][:,:,icol].data)

# assign qlrs
dum_var = model_data['RAINQM'][:,:,icol]*0.
model_data = model_data.assign(qlrs = dum_var + model_data['RAINQM'][:,:,icol].data)

# assign qics
dum_var = model_data['CLDICE'][:,:,icol]*0.
model_data = model_data.assign(qics = dum_var + model_data['CLDICE'][:,:,icol].data)

# assign qips
dum_var = model_data['SNOWQM'][:,:,icol]*0.
model_data = model_data.assign(qips = dum_var + model_data['SNOWQM'][:,:,icol].data)

# assign nlcs
dum_var = model_data['NUMLIQ'][:,:,icol]*0.
model_data = model_data.assign(nlcs = dum_var + model_data['NUMLIQ'][:,:,icol].data)

# assign nlrs
dum_var = model_data['NUMRAI'][:,:,icol]*0.
model_data = model_data.assign(nlrs = dum_var + model_data['NUMRAI'][:,:,icol].data)

# assign nics
dum_var = model_data['NUMICE'][:,:,icol]*0.
model_data = model_data.assign(nics = dum_var + model_data['NUMICE'][:,:,icol].data)

# assign nips
dum_var = model_data['NUMSNO'][:,:,icol]*0.
model_data = model_data.assign(nips = dum_var + model_data['NUMSNO'][:,:,icol].data)

# assign flcs
dum_var = model_data['FREQL'][:,:,icol]*0.
model_data = model_data.assign(flcs = dum_var + model_data['FREQL'][:,:,icol].data)

# assign flrs
dum_var = model_data['FREQR'][:,:,icol]*0.
model_data = model_data.assign(flrs = dum_var + model_data['FREQR'][:,:,icol].data)

# assign fics
dum_var = model_data['FREQI'][:,:,icol]*0.
model_data = model_data.assign(fics = dum_var + model_data['FREQI'][:,:,icol].data)

# assign fips
dum_var = model_data['FREQS'][:,:,icol]*0.
model_data = model_data.assign(fips = dum_var + model_data['FREQS'][:,:,icol].data)

# assign vmlcs
dum_var = model_data['VTRMC'][:,:,icol]*0.
model_data = model_data.assign(vmlcs = dum_var + model_data['VTRMC'][:,:,icol].data)

# assign vmlrs
dum_var = model_data['UMR'][:,:,icol]*0.
model_data = model_data.assign(vmlrs = dum_var + model_data['UMR'][:,:,icol].data)

# assign vmics
dum_var = model_data['VTRMI'][:,:,icol]*0.
model_data = model_data.assign(vmics = dum_var + model_data['VTRMI'][:,:,icol].data)

# assign vmips
dum_var = model_data['UMS'][:,:,icol]*0.
model_data = model_data.assign(vmips = dum_var + model_data['UMS'][:,:,icol].data)

# assign height
dum_var = model_data['Z3'][:,:,icol]*0.
model_data = model_data.assign(zf = dum_var + model_data['Z3'][:,:,icol].data)

# assign ua
dum_var = model_data['U'][:,:,icol]*0.
model_data = model_data.assign(ua = dum_var + model_data['U'][:,:,icol].data)

# assign va
dum_var = model_data['V'][:,:,icol]*0.
model_data = model_data.assign(va = dum_var + model_data['V'][:,:,icol].data)

# assign ta
dum_var = model_data['T'][:,:,icol]*0.
model_data = model_data.assign(ta = dum_var + model_data['T'][:,:,icol].data)

# assign qv
dum_var = model_data['Q'][:,:,icol]*0.
model_data = model_data.assign(qv = dum_var + model_data['Q'][:,:,icol].data)

# assign hur
dum_var = model_data['RHW'][:,:,icol]*0.
model_data = model_data.assign(hur = dum_var + model_data['RHW'][:,:,icol].data / 1e2)

# assign huri
dum_var = model_data['RHI'][:,:,icol]*0.
model_data = model_data.assign(huri = dum_var + model_data['RHI'][:,:,icol].data / 1e2)

# calculate theta
dum_var = model_data['T'][:,:,icol]*0.
pmid = model_data['hyam'].data * model_data['P0'].data + model_data['hybm'].data * model_data['PS'].data
dum_theta = model_data['T'][:,:,0].data * (1e5 / pmid) ** 0.286
model_data = model_data.assign(theta = dum_var + dum_theta)

# assign qlc
dum_var = model_data['CLDLIQ'][:,:,icol]*0.
model_data = model_data.assign(qlc = dum_var + model_data['CLDLIQ'][:,:,icol].data)

# assign qlr
dum_var = model_data['RAINQM'][:,:,icol]*0.
model_data = model_data.assign(qlr = dum_var + model_data['RAINQM'][:,:,icol].data)

# assign qi
dum_var = model_data['CLDICE'][:,:,icol]*0.
model_data = model_data.assign(qi = dum_var + model_data['CLDICE'][:,:,icol].data + model_data['SNOWQM'][:,:,icol].data)

# assign fh
dum_var = model_data['CLOUD'][:,:,icol]*0.
model_data = model_data.assign(fh = dum_var + model_data['CLOUD'][:,:,icol].data)

# assign flc
dum_var = model_data['FREQL'][:,:,icol]*0.
model_data = model_data.assign(flc = dum_var + model_data['FREQL'][:,:,icol].data)

# assign fc
dum_var = model_data['CONCLD'][:,:,icol]*0.
model_data = model_data.assign(fc = dum_var + model_data['CONCLD'][:,:,icol].data)

# calculate zonal momentum flux
dum_var = model_data['T'][:,:,icol]*0.
for i in np.arange(len(dum_var[:,0])):
    f_int = interpolate.interp1d(pint[i,:], model_data['UPWP_CLUBB'][i,:,0].data, kind='linear', fill_value='extrapolate')
    dum_var[i,:] = f_int(pmid[i,:])
model_data = model_data.assign(uw = dum_var)

# calculate meridional momentum flux
dum_var = model_data['T'][:,:,icol]*0.
for i in np.arange(len(dum_var[:,0])):
    f_int = interpolate.interp1d(pint[i,:], model_data['VPWP_CLUBB'][i,:,0].data, kind='linear', fill_value='extrapolate')
    dum_var[i,:] = f_int(pmid[i,:])
model_data = model_data.assign(vw = dum_var)

# calculate variance of upward air velocity
dum_var = model_data['T'][:,:,icol]*0.
for i in np.arange(len(dum_var[:,0])):
    f_int = interpolate.interp1d(pint[i,:], model_data['WP2_CLUBB'][i,:,0].data, kind='linear', fill_value='extrapolate')
    dum_var[i,:] = f_int(pmid[i,:])
model_data = model_data.assign(w2 = dum_var)

# calculate downwelling longwave flux in air
dum_var = model_data['T'][:,:,icol]*0.
for i in np.arange(len(dum_var[:,0])):
    f_int = interpolate.interp1d(pint[i,:], model_data['FDL'][i,:,0].data, kind='linear', fill_value='extrapolate')
    dum_var[i,:] = f_int(pmid[i,:])
model_data = model_data.assign(rld = dum_var)

# calculate downwelling longwave flux in air
dum_var = model_data['T'][:,:,icol]*0.
for i in np.arange(len(dum_var[:,0])):
    f_int = interpolate.interp1d(pint[i,:], model_data['FUL'][i,:,0].data, kind='linear', fill_value='extrapolate')
    dum_var[i,:] = f_int(pmid[i,:])
model_data = model_data.assign(rlu = dum_var)

# assign rhoa
dum_var = model_data['T'][:,:,icol]*0.
for i in np.arange(len(dum_var[:,0])):
    f_int = interpolate.interp1d(pint[i,:], model_data['RHO_CLUBB'][i,:,0].data, kind='linear', fill_value='extrapolate')
    dum_var[i,:] = f_int(pmid[i,:])
model_data = model_data.assign(rhoa = dum_var)

# assign tke
dum_var = model_data['T'][:,:,icol]*0.
dum_tke = 0.5 * (model_data['UP2_CLUBB'].data + model_data['VP2_CLUBB'].data + model_data['WP2_CLUBB'].data)
for i in np.arange(len(dum_var[:,0])):
    f_int = interpolate.interp1d(pint[i,:], dum_tke[i,:,0].data, kind='linear', fill_value='extrapolate')
    dum_var[i,:] = f_int(pmid[i,:])
model_data = model_data.assign(tke = dum_var)

# calculate inversioin height
dum_var = model_data['PBLH'][:,icol]*0.
dum_dtdp = (np.roll(model_data['theta'], 1, axis=1)[:,1:] - model_data['theta'][:,1:]) / (np.roll(pmid, 1, axis=1)[:,1:] - pmid[:,1:])
dum_dtdp = np.where(pmid[:,1:] > 40000., dum_dtdp, np.nan)
indmin = np.nanargmin(dum_dtdp,axis=1)
for i in np.arange(len(indmin)):
    dum_var[i] = model_data['Z3'][i,indmin[i]+1,icol].data
model_data = model_data.assign(zi = dum_var)

Prepare output file in DEPHY format#

Read requested variables list#

Variable description, naming, units, and dimensions.

# read list of requested variables
vars_mean_list = pd.read_excel('https://docs.google.com/spreadsheets/d/1Vl8jYGviet7EtXZuQiitrx4NSkV1x27aJAhxxjBb9zI/export?gid=1026157027&format=xlsx',
                              sheet_name='SCM')
pd.set_option('display.max_rows', None)
vars_mean_list
standard_name variable_id units dimensions comment (reported at end of each model physics time step, green=minimum, red=granularity enabling EMC2)
0 time time s – dimension, seconds since 2020-03-12 18:00:00
1 pressure_layer layer 1 – dimension, pressure layer number from 1 at sur...
2 air_pressure pa Pa time, layer pressure at mid-level points (native model lev...
3 layer_top_pressure pe Pa time, layer dimension, pressure at layer top points (used ...
4 surface_pressure ps Pa time –
5 surface_temperature ts K time –
6 surface_friction_velocity ustar m s-1 time –
7 surface_roughness_length_for_momentum_in_air z0 m time –
8 surface_roughness_length_for_heat_in_air z0h m time –
9 surface_roughness_length_for_humidity_in_air z0q m time –
10 surface_upward_sensible_heat_flux hfss W m-2 time –
11 surface_upward_latent_heat_flux hfls W m-2 time –
12 obukhov_length ol m time –
13 pbl_height pblh m time PBL scheme layer thickness (if available)
14 inversion_height zi m time sharpest vertical gradient in air_potential_te...
15 atmosphere_mass_content_of_liquid_cloud_water lwpc kg m-2 time scene (all sky); cloud water path in all class...
16 atmosphere_mass_content_of_rain_water lwpr kg m-2 time scene (all sky); rain water path in all classe...
17 atmosphere_mass_content_of_ice_water iwp kg m-2 time scene (all sky); all ice-phase hydrometeors in...
18 area_fraction_cover_of_hydrometeors cf 1 time all hydrometeors and cloud types (e.g., all ph...
19 area_fraction_cover_of_liquid_cloud cflc 1 time liquid cloud cover without precipitation, incl...
20 area_fraction_cover_of_convective_hydrometeors cfc 1 time all hydrometeors, default breakdown into conve...
21 optical_depth od 1 time scene (all sky); mid-visible, all hydrometeors...
22 optical_depth_of_liquid_cloud odlc 1 time scene (all sky); mid-visible, cloud liquid onl...
23 precipitation_flux_at_surface pr kg m-2 s-1 time scene (all sky); all hydrometeors
24 precipitation_flux_of_ice_at_surface pri kg m-2 s-1 time scene (all sky); all ice phase hydrometeors
25 toa_outgoing_longwave_flux rlut W m-2 time –
26 surface_downwelling_longwave_flux rlds W m-2 time –
27 surface_upwelling_longwave_flux rlus W m-2 time –
28 surface_sea_spray_number_flux ssaf m-2 s-1 time when using prognostic aerosol; emission only (...
29 height zf m time, layer altitude of layer mid-level points above sea s...
30 eastward_wind ua m s-1 time, layer –
31 northward_wind va m s-1 time, layer –
32 air_dry_density rhoa kg m-3 time, layer per kg dry air
33 air_temperature ta K time, layer –
34 water_vapor_mixing_ratio qv kg kg-1 time, layer –
35 relative_humidity hur 1 time, layer relative to liquid
36 relative_humidity_over_ice huri 1 time, layer relative to ice
37 air_potential_temperature theta K time, layer –
38 mass_mixing_ratio_of_cloud_liquid_water_in_air qlc kg kg-1 time, layer scene (all sky) per kg dry air; cloud water pa...
39 mass_mixing_ratio_of_rain_water_in_air qlr kg kg-1 time, layer rain water path only in all classes (e.g., con...
40 mass_mixing_ratio_of_ice_water_in_air qi kg kg-1 time, layer all ice water path in all classes (e.g., conve...
41 area_fraction_of_hydrometeors fh 1 time, layer all hydrometeors and cloud types (e.g., all ph...
42 area_fraction_of_liquid_cloud flc 1 time, layer liquid cloud cover without precipitation, incl...
43 area_fraction_of_convective_hydrometeors fc 1 time, layer all hydrometeors, default breakdown into conve...
44 precipitation_flux_in_air prf kg m-2 s-1 time, layer scene (all sky); all hydrometeors
45 precipitation_flux_in_air_in_ice_phase prfi kg m-2 s-1 time, layer scene (all sky); all ice phase hydrometeors
46 specific_turbulent_kinetic_energy tke m2 s-2 time, layer –
47 dissipation_rate_of_turbulent_kinetic_energy eps m2 s-3 time, layer report as negative
48 zonal_momentum_flux uw m2 s-2 time, layer parameterized turbulent flux
49 meridional_momentum_flux vw m2 s-2 time, layer parameterized turbulent flux
50 variance_of_upward_air_velocity w2 m2 s-2 time, layer parameterized turbulent flux
51 vertical_flux_potential_temperature wth K m s-1 time, layer parameterized turbulent flux
52 vertical_flux_liquid_ice_water_potential_tempe... vf_thli K m s-1 time, layer parameterized turbulent flux; include sediment...
53 vertical_flux_water_vapor wqv kg kg-1 m s-1 time, layer parameterized turbulent flux
54 vertical_flux_total_water vf_qt kg kg-1 m s-1 time, layer parameterized turbulent flux; vapor+all liquid...
55 convection_updraft_mass_flux cmfu kg m-2 s-1 time, layer –
56 convection_downdraft_mass_flux cmfd kg m-2 s-1 time, layer –
57 downwelling_longwave_flux_in_air rld W m-2 time, layer –
58 upwelling_longwave_flux_in_air rlu W m-2 time, layer –
59 tendency_of_air_potential_temperature_due_to_r... dth_rad K s-1 time, layer scene (all sky)
60 tendency_of_air_potential_temperature_due_to_m... dth_micro K s-1 time, layer including net condensation and precipitation i...
61 tendency_of_air_potential_temperature_due_to_m... dth_turb K s-1 time, layer including surface fluxes
62 tendency_of_water_vapor_mixing_ratio_due_to_mi... dq_micro s-1 time, layer including net condensation and precipitation i...
63 tendency_of_water_vapor_mixing_ratio_due_to_mi... dq_turb s-1 time, layer including surface fluxes
64 number_of_total_aerosol_mode1 na1 kg-1 time, layer when using prognostic aerosol; scene (all sky)...
65 number_of_total_aerosol_mode2 na2 kg-1 time, layer accumulation mode
66 number_of_total_aerosol_mode3 na3 kg-1 time, layer sea spray mode
67 tendency_of_aerosol_number_due_to_warm_microph... dna_micro_warm kg-1 s-1 time, layer activated and unactivated aerosol (sum over al...
68 tendency_of_aerosol_number_due_to_cold_microph... dna_micro_cold kg-1 s-1 time, layer activated and unactivated aerosol (sum over al...
69 tendency_of_aerosol_number_due_to_mixing dna_turb kg-1 s-1 time, layer activated and unactivated aerosol (sum over al...
70 tendency_of_ice_number_due_to_heterogeneous_fr... dni_het kg-1 s-1 time, layer sum of primary ice nucleation due to activatio...
71 tendency_of_ice_number_due_to_secondary_ice_pr... dni_sip kg-1 s-1 time, layer sum of secondary ice formation processes (e.g....
72 tendency_of_ice_number_due_to_homogeneous_free... dni_hom kg-1 s-1 time, layer ice nucleation source due to homogoeneous free...
73 mass_mixing_ratio_of_liquid_cloud_water_in_air... qlcs kg kg-1 time, layer scene (all sky) per kg dry air; default breakd...
74 mass_mixing_ratio_of_rain_water_in_air_stratiform qlrs kg kg-1 time, layer –
75 mass_mixing_ratio_of_ice_cloud_in_air_stratiform qics kg kg-1 time, layer default breakdown as for liquid; if other ice-...
76 mass_mixing_ratio_of_ice_precipitation_in_air_... qips kg kg-1 time, layer –
77 mass_mixing_ratio_of_liquid_cloud_water_in_air... qlcc kg kg-1 time, layer –
78 mass_mixing_ratio_of_rain_water_in_air_convective qlrc kg kg-1 time, layer –
79 mass_mixing_ratio_of_ice_cloud_in_air_convective qicc kg kg-1 time, layer –
80 mass_mixing_ratio_of_ice_precipitation_in_air_... qipc kg kg-1 time, layer –
81 number_of_liquid_cloud_droplets_in_air_stratiform nlcs kg-1 time, layer scene (all sky) per kg dry air; if other categ...
82 number_of_rain_drops_in_air_stratiform nlrs kg-1 time, layer –
83 number_of_ice_cloud_crystals_in_air_stratiform nics kg-1 time, layer if other ice-phase categories are used, provid...
84 number_of_ice_precipitation_crystals_in_air_st... nips kg-1 time, layer –
85 effective_radius_of_liquid_cloud_droplets_conv... relcc m time, layer EMC2 uses effective radius for any hydrometeor...
86 effective_radius_of_rain_convective relrc m time, layer –
87 effective_radius_of_ice_cloud_convective reicc m time, layer –
88 effective_radius_of_ice_precipitation_convective reipc m time, layer –
89 area_fraction_of_liquid_cloud_stratiform flcs 1 time, layer EMC2 uses area fraction profiles for all hydro...
90 area_fraction_of_rain_stratiform flrs 1 time, layer –
91 area_fraction_of_ice_cloud_stratiform fics 1 time, layer if other ice categories are used, provide addi...
92 area_fraction_of_ice_precipitation_stratiform fips 1 time, layer –
93 area_fraction_of_liquid_cloud_convective flcc 1 time, layer –
94 area_fraction_of_rain_convective flrc 1 time, layer –
95 area_fraction_of_ice_cloud_convective ficc 1 time, layer –
96 area_fraction_of_ice_precipitation_convective fipc 1 time, layer –
97 mass_weighted_fall_speed_of_liquid_cloud_water... vmlcs m s-1 time, layer EMC2 uses mass-weighted fall-speed profiles fo...
98 mass_weighted_fall_speed_of_rain_stratiform vmlrs m s-1 time, layer –
99 mass_weighted_fall_speed_of_ice_cloud_stratiform vmics m s-1 time, layer if other ice-phase categories are used, provid...
100 mass_weighted_fall_speed_of_ice_precipitation_... vmips m s-1 time, layer –
101 mass_weighted_fall_speed_of_liquid_cloud_water... vmlcc m s-1 time, layer –
102 mass_weighted_fall_speed_of_rain_convective vmlrc m s-1 time, layer –
103 mass_weighted_fall_speed_of_cloud_ice_crystals... vmicc m s-1 time, layer –
104 mass_weighted_fall_speed_of_ice_precipitation_... vmipc m s-1 time, layer –

Match E3SM variables to requested outputs#

Expand the table to include columns that indicate E3SM model variable names and any conversion factor.

# drop comments
vars_mean_list = vars_mean_list.drop(columns='comment (reported at end of each model physics time step, green=minimum, red=granularity enabling EMC2)')

# add columns to contain model output name and units conversion factors
vars_mean_list = vars_mean_list.assign(model_name='missing data',conv_factor=1.0)

# match to E3SM variable names and specify conversion factors
for index in vars_mean_list.index:
    standard_name = vars_mean_list.standard_name.iat[index]
    if standard_name=='air_pressure': 
        vars_mean_list.model_name.iat[index] = 'pa'
    if standard_name=='layer_top_pressure': 
        vars_mean_list.model_name.iat[index] = 'pe'
    if standard_name=='surface_pressure': 
        vars_mean_list.model_name.iat[index] = 'ps'
        vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='surface_temperature': 
        vars_mean_list.model_name.iat[index] = 'ts'
    if standard_name=='surface_friction_velocity': 
        vars_mean_list.model_name.iat[index] = 'ustar'
#    if standard_name=='surface_roughness_length_for_momentum_in_air': 
#        vars_mean_list.model_name.iat[index] = 'z0m'
#    if standard_name=='surface_roughness_length_for_heat_in_air': 
#        vars_mean_list.model_name.iat[index] = 'z0h'
#    if standard_name=='surface_roughness_length_for_humidity_in_air': 
#        vars_mean_list.model_name.iat[index] = 'z0q'
    if standard_name=='surface_upward_sensible_heat_flux': 
        vars_mean_list.model_name.iat[index] = 'hfss'
        vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='surface_upward_latent_heat_flux': 
        vars_mean_list.model_name.iat[index] = 'hfls'
        vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='obukhov_length': 
        vars_mean_list.model_name.iat[index] = 'ol'
    if standard_name=='pbl_height': 
        vars_mean_list.model_name.iat[index] = 'pblh'
    if standard_name=='inversion_height': 
        vars_mean_list.model_name.iat[index] = 'zi'
    if standard_name=='atmosphere_mass_content_of_liquid_cloud_water': 
        vars_mean_list.model_name.iat[index] = 'lwpc'
        vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='atmosphere_mass_content_of_rain_water': 
        vars_mean_list.model_name.iat[index] = 'lwpr'
        vars_mean_list.conv_factor.iat[index] = 1.
    if do_ice:
        if standard_name=='atmosphere_mass_content_of_ice_water': 
            vars_mean_list.model_name.iat[index] = 'iwp'
            vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='area_fraction_cover_of_hydrometeors': 
        vars_mean_list.model_name.iat[index] = 'cf'
#    if standard_name=='area_fraction_cover_of_liquid_cloud': 
#        vars_mean_list.model_name.iat[index] = ''
    if standard_name=='area_fraction_cover_of_convective_hydrometeors': 
        vars_mean_list.model_name.iat[index] = 'cfc'
    # if standard_name=='optical_depth': 
    #     vars_mean_list.model_name.iat[index] = 'od'
    # if standard_name=='optical_depth_of_liquid_water': 
    #     vars_mean_list.model_name.iat[index] = 'odlc'
    if standard_name=='precipitation_flux_at_surface': 
        vars_mean_list.model_name.iat[index] = 'pr'
        vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='precipitation_flux_of_ice_at_surface': 
        vars_mean_list.model_name.iat[index] = 'pri'
        vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='toa_outgoing_longwave_flux': 
        vars_mean_list.model_name.iat[index] = 'rlut'
    if standard_name=='surface_downwelling_longwave_flux': 
        vars_mean_list.model_name.iat[index] = 'rlds'  
    if standard_name=='surface_upwelling_longwave_flux': 
        vars_mean_list.model_name.iat[index] = 'rlus'  
    if standard_name=='height': 
        vars_mean_list.model_name.iat[index] = 'zf'
    if standard_name=='eastward_wind': 
        vars_mean_list.model_name.iat[index] = 'ua'
    if standard_name=='northward_wind': 
        vars_mean_list.model_name.iat[index] = 'va'
    if standard_name=='air_dry_density': 
        vars_mean_list.model_name.iat[index] = 'rhoa'
    if standard_name=='air_temperature': 
        vars_mean_list.model_name.iat[index] = 'ta'
    if standard_name=='water_vapor_mixing_ratio': 
        vars_mean_list.model_name.iat[index] = 'qv'
    if standard_name=='relative_humidity': 
        vars_mean_list.model_name.iat[index] = 'hur'
        vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='relative_humidity_over_ice': 
        vars_mean_list.model_name.iat[index] = 'huri'
        vars_mean_list.conv_factor.iat[index] = 1.
    if standard_name=='air_potential_temperature': 
        vars_mean_list.model_name.iat[index] = 'theta'
    if standard_name=='mass_mixing_ratio_of_cloud_liquid_water_in_air': 
        vars_mean_list.model_name.iat[index] = 'qlc'
    if standard_name=='mass_mixing_ratio_of_rain_water_in_air': 
        vars_mean_list.model_name.iat[index] = 'qlr'
    if do_ice: 
        if standard_name=='mass_mixing_ratio_of_ice_water_in_air': 
            vars_mean_list.model_name.iat[index] = 'qi'
    if standard_name=='area_fraction_of_hydrometeors': 
        vars_mean_list.model_name.iat[index] = 'fh'
    if standard_name=='area_fraction_of_liquid_cloud': 
        vars_mean_list.model_name.iat[index] = 'flc'
    if standard_name=='area_fraction_of_convective_hydrometeors': 
        vars_mean_list.model_name.iat[index] = 'fc'
    # if standard_name=='precipitation_flux_in_air': 
    #     vars_mean_list.model_name.iat[index] = 'prt'
    #     vars_mean_list.conv_factor.iat[index] = 1./86400
    # if do_ice:
    #     if standard_name=='precipitation_flux_in_air_in_ice_phase': 
    #         vars_mean_list.model_name.iat[index] = 'pit'
    #         vars_mean_list.conv_factor.iat[index] = 1./86400
    if standard_name=='specific_turbulent_kinetic_energy': 
        vars_mean_list.model_name.iat[index] = 'tke'
    # if standard_name=='disspation_rate_of_turbulent_kinetic_energy': 
    #     vars_mean_list.model_name.iat[index] = 'dissip_tke_turb'
    #     vars_mean_list.conv_factor.iat[index] = -1.
    if standard_name=='zonal_momentum_flux': 
        vars_mean_list.model_name.iat[index] = 'uw'
    if standard_name=='meridional_momentum_flux': 
        vars_mean_list.model_name.iat[index] = 'vw'
    if standard_name=='variance_of_upward_air_velocity': 
        vars_mean_list.model_name.iat[index] = 'w2'
    # if standard_name=='vertical_flux_potential_temperature': 
    #     vars_mean_list.model_name.iat[index] = 'wth_turb'
#    if standard_name=='vertical_flux_liquid_water_potential_temperature': 
#        vars_mean_list.model_name.iat[index] = ''
    # if standard_name=='vertical_flux_water_vapor': 
    #     vars_mean_list.model_name.iat[index] = 'wq_turb'
    # if standard_name=='vertical_flux_total_water': 
    #     vars_mean_list.model_name.iat[index] = 'wqt_turb'
    # if standard_name=='convection_updraft_mass_flux': 
    #     vars_mean_list.model_name.iat[index] = 'lwdp'
    # if standard_name=='convection_downdraft_mass_flux': 
    #     vars_mean_list.model_name.iat[index] = 'lwdp'
    if standard_name=='downwelling_longwave_flux_in_air': 
        vars_mean_list.model_name.iat[index] = 'rld'
    if standard_name=='upwelling_longwave_flux_in_air': 
        vars_mean_list.model_name.iat[index] = 'rlu'
    # if standard_name=='tendency_of_air_potential_temperature_due_to_radiative_heating': 
    #     vars_mean_list.model_name.iat[index] = 'dth_lw'
    #     vars_mean_list.conv_factor.iat[index] = 1./86400
    # if standard_name=='tendency_of_air_potential_temperature_due_to_microphysics': 
    #     vars_mean_list.model_name.iat[index] = 'dth_micro'
    #     vars_mean_list.conv_factor.iat[index] = 1./86400
    # if standard_name=='tendency_of_air_potential_temperature_due_to_mixing': 
    #     vars_mean_list.model_name.iat[index] = 'dth_turb'
    #     vars_mean_list.conv_factor.iat[index] = 1./86400
    # if standard_name=='tendency_of_water_vapor_mixing_ratio_due_to_microphysics': 
    #     vars_mean_list.model_name.iat[index] = 'dq_micro'
    #     vars_mean_list.conv_factor.iat[index] = 1./86400
    # if standard_name=='tendency_of_water_vapor_mixing_ratio_due_to_mixing': 
    #     vars_mean_list.model_name.iat[index] = 'dq_turb'
    #     vars_mean_list.conv_factor.iat[index] = 1./86400
    if standard_name=='mass_mixing_ratio_of_liquid_cloud_water_in_air_stratiform': 
        vars_mean_list.model_name.iat[index] = 'qlcs'
    if standard_name=='mass_mixing_ratio_of_rain_water_in_air_stratiform': 
        vars_mean_list.model_name.iat[index] = 'qlrs'
    if do_ice:
        if standard_name=='mass_mixing_ratio_of_ice_cloud_in_air_stratiform': 
            vars_mean_list.model_name.iat[index] = 'qics'
        if standard_name=='mass_mixing_ratio_of_ice_precipitation_in_air_stratiform': 
            vars_mean_list.model_name.iat[index] = 'qips'
    # if standard_name=='mass_mixing_ratio_of_liquid_cloud_water_in_air_convective': 
    #     vars_mean_list.model_name.iat[index] = 'QCLmc'
    # if standard_name=='mass_mixing_ratio_of_rain_water_in_air_convective': 
    #     vars_mean_list.model_name.iat[index] = 'QPLmc'
    # if do_ice:
    #     if standard_name=='mass_mixing_ratio_of_ice_cloud_in_air_convective': 
    #         vars_mean_list.model_name.iat[index] = 'QCImc'
    #     if standard_name=='mass_mixing_ratio_of_ice_precipitation_in_air_convective': 
    #         vars_mean_list.model_name.iat[index] = 'QPImc'
    if standard_name=='number_of_liquid_cloud_droplets_in_air_stratiform': 
        vars_mean_list.model_name.iat[index] = 'nlcs'
    if standard_name=='number_of_rain_drops_in_air_stratiform': 
        vars_mean_list.model_name.iat[index] = 'nlrs'
    if do_ice:
        if standard_name=='number_of_ice_cloud_crystals_in_air_stratiform': 
            vars_mean_list.model_name.iat[index] = 'nics'
        if standard_name=='number_of_ice_precipitation_crystals_in_air_stratiform': 
            vars_mean_list.model_name.iat[index] = 'nips'
    # if standard_name=='effective_radius_of_liquid_cloud_droplets_convective': 
    #     vars_mean_list.model_name.iat[index] = 're_mccl'
    # if standard_name=='effective_radius_of_rain_convective': 
    #     vars_mean_list.model_name.iat[index] = 're_mcpl'
    # if do_ice:
    #     if standard_name=='effective_radius_of_ice_cloud_convective': 
    #         vars_mean_list.model_name.iat[index] = 're_mcci'
    #     if standard_name=='effective_radius_of_ice_precipitation_convective': 
    #         vars_mean_list.model_name.iat[index] = 're_mcpi'
    if standard_name=='area_fraction_of_liquid_cloud_stratiform': 
        vars_mean_list.model_name.iat[index] = 'flcs'
    if standard_name=='area_fraction_of_rain_stratiform': 
        vars_mean_list.model_name.iat[index] = 'flrs'
    if do_ice:
        if standard_name=='area_fraction_of_ice_cloud_stratiform': 
            vars_mean_list.model_name.iat[index] = 'fics'
        if standard_name=='area_fraction_of_ice_precipitation_stratiform': 
            vars_mean_list.model_name.iat[index] = 'fips'
    # if standard_name=='area_fraction_of_liquid_cloud_convective': 
    #     vars_mean_list.model_name.iat[index] = 'cldmccl'
    # if standard_name=='area_fraction_of_rain_convective': 
    #     vars_mean_list.model_name.iat[index] = 'cldmcpl'
    # if do_ice:
    #     if standard_name=='area_fraction_of_ice_cloud_convective': 
    #         vars_mean_list.model_name.iat[index] = 'cldmcci'
    #     if standard_name=='area_fraction_of_ice_precipitation_convective': 
    #         vars_mean_list.model_name.iat[index] = 'cldmcpi'
    if standard_name=='mass_weighted_fall_speed_of_liquid_cloud_water_stratiform': 
        vars_mean_list.model_name.iat[index] = 'vmlcs'
    if standard_name=='mass_weighted_fall_speed_of_rain_stratiform': 
        vars_mean_list.model_name.iat[index] = 'vmlrs'
    if do_ice:
        if standard_name=='mass_weighted_fall_speed_of_ice_cloud_stratiform': 
            vars_mean_list.model_name.iat[index] = 'vmics'
        if standard_name=='mass_weighted_fall_speed_of_ice_precipitation_stratiform': 
            vars_mean_list.model_name.iat[index] = 'vmips'
    # if standard_name=='mass_weighted_fall_speed_of_liquid_cloud_water_convective': 
    #     vars_mean_list.model_name.iat[index] = 'vm_mccl'
    # if standard_name=='mass_weighted_fall_speed_of_rain_convective': 
    #     vars_mean_list.model_name.iat[index] = 'vm_mcpl'
    # if do_ice:
    #     if standard_name=='mass_weighted_fall_speed_of_cloud_ice_crystals_convective': 
    #         vars_mean_list.model_name.iat[index] = 'vm_mcci'
    #     if standard_name=='mass_weighted_fall_speed_of_ice_precipitation_convective': 
    #         vars_mean_list.model_name.iat[index] = 'vm_mcpi'

vars_mean_list[2:] # echo variables (first two rows are dimensions)
standard_name variable_id units dimensions model_name conv_factor
2 air_pressure pa Pa time, layer pa 1.0
3 layer_top_pressure pe Pa time, layer pe 1.0
4 surface_pressure ps Pa time ps 1.0
5 surface_temperature ts K time ts 1.0
6 surface_friction_velocity ustar m s-1 time ustar 1.0
7 surface_roughness_length_for_momentum_in_air z0 m time missing data 1.0
8 surface_roughness_length_for_heat_in_air z0h m time missing data 1.0
9 surface_roughness_length_for_humidity_in_air z0q m time missing data 1.0
10 surface_upward_sensible_heat_flux hfss W m-2 time hfss 1.0
11 surface_upward_latent_heat_flux hfls W m-2 time hfls 1.0
12 obukhov_length ol m time ol 1.0
13 pbl_height pblh m time pblh 1.0
14 inversion_height zi m time zi 1.0
15 atmosphere_mass_content_of_liquid_cloud_water lwpc kg m-2 time lwpc 1.0
16 atmosphere_mass_content_of_rain_water lwpr kg m-2 time lwpr 1.0
17 atmosphere_mass_content_of_ice_water iwp kg m-2 time missing data 1.0
18 area_fraction_cover_of_hydrometeors cf 1 time cf 1.0
19 area_fraction_cover_of_liquid_cloud cflc 1 time missing data 1.0
20 area_fraction_cover_of_convective_hydrometeors cfc 1 time cfc 1.0
21 optical_depth od 1 time missing data 1.0
22 optical_depth_of_liquid_cloud odlc 1 time missing data 1.0
23 precipitation_flux_at_surface pr kg m-2 s-1 time pr 1.0
24 precipitation_flux_of_ice_at_surface pri kg m-2 s-1 time pri 1.0
25 toa_outgoing_longwave_flux rlut W m-2 time rlut 1.0
26 surface_downwelling_longwave_flux rlds W m-2 time rlds 1.0
27 surface_upwelling_longwave_flux rlus W m-2 time rlus 1.0
28 surface_sea_spray_number_flux ssaf m-2 s-1 time missing data 1.0
29 height zf m time, layer zf 1.0
30 eastward_wind ua m s-1 time, layer ua 1.0
31 northward_wind va m s-1 time, layer va 1.0
32 air_dry_density rhoa kg m-3 time, layer rhoa 1.0
33 air_temperature ta K time, layer ta 1.0
34 water_vapor_mixing_ratio qv kg kg-1 time, layer qv 1.0
35 relative_humidity hur 1 time, layer hur 1.0
36 relative_humidity_over_ice huri 1 time, layer huri 1.0
37 air_potential_temperature theta K time, layer theta 1.0
38 mass_mixing_ratio_of_cloud_liquid_water_in_air qlc kg kg-1 time, layer qlc 1.0
39 mass_mixing_ratio_of_rain_water_in_air qlr kg kg-1 time, layer qlr 1.0
40 mass_mixing_ratio_of_ice_water_in_air qi kg kg-1 time, layer missing data 1.0
41 area_fraction_of_hydrometeors fh 1 time, layer fh 1.0
42 area_fraction_of_liquid_cloud flc 1 time, layer flc 1.0
43 area_fraction_of_convective_hydrometeors fc 1 time, layer fc 1.0
44 precipitation_flux_in_air prf kg m-2 s-1 time, layer missing data 1.0
45 precipitation_flux_in_air_in_ice_phase prfi kg m-2 s-1 time, layer missing data 1.0
46 specific_turbulent_kinetic_energy tke m2 s-2 time, layer tke 1.0
47 dissipation_rate_of_turbulent_kinetic_energy eps m2 s-3 time, layer missing data 1.0
48 zonal_momentum_flux uw m2 s-2 time, layer uw 1.0
49 meridional_momentum_flux vw m2 s-2 time, layer vw 1.0
50 variance_of_upward_air_velocity w2 m2 s-2 time, layer w2 1.0
51 vertical_flux_potential_temperature wth K m s-1 time, layer missing data 1.0
52 vertical_flux_liquid_ice_water_potential_tempe... vf_thli K m s-1 time, layer missing data 1.0
53 vertical_flux_water_vapor wqv kg kg-1 m s-1 time, layer missing data 1.0
54 vertical_flux_total_water vf_qt kg kg-1 m s-1 time, layer missing data 1.0
55 convection_updraft_mass_flux cmfu kg m-2 s-1 time, layer missing data 1.0
56 convection_downdraft_mass_flux cmfd kg m-2 s-1 time, layer missing data 1.0
57 downwelling_longwave_flux_in_air rld W m-2 time, layer rld 1.0
58 upwelling_longwave_flux_in_air rlu W m-2 time, layer rlu 1.0
59 tendency_of_air_potential_temperature_due_to_r... dth_rad K s-1 time, layer missing data 1.0
60 tendency_of_air_potential_temperature_due_to_m... dth_micro K s-1 time, layer missing data 1.0
61 tendency_of_air_potential_temperature_due_to_m... dth_turb K s-1 time, layer missing data 1.0
62 tendency_of_water_vapor_mixing_ratio_due_to_mi... dq_micro s-1 time, layer missing data 1.0
63 tendency_of_water_vapor_mixing_ratio_due_to_mi... dq_turb s-1 time, layer missing data 1.0
64 number_of_total_aerosol_mode1 na1 kg-1 time, layer missing data 1.0
65 number_of_total_aerosol_mode2 na2 kg-1 time, layer missing data 1.0
66 number_of_total_aerosol_mode3 na3 kg-1 time, layer missing data 1.0
67 tendency_of_aerosol_number_due_to_warm_microph... dna_micro_warm kg-1 s-1 time, layer missing data 1.0
68 tendency_of_aerosol_number_due_to_cold_microph... dna_micro_cold kg-1 s-1 time, layer missing data 1.0
69 tendency_of_aerosol_number_due_to_mixing dna_turb kg-1 s-1 time, layer missing data 1.0
70 tendency_of_ice_number_due_to_heterogeneous_fr... dni_het kg-1 s-1 time, layer missing data 1.0
71 tendency_of_ice_number_due_to_secondary_ice_pr... dni_sip kg-1 s-1 time, layer missing data 1.0
72 tendency_of_ice_number_due_to_homogeneous_free... dni_hom kg-1 s-1 time, layer missing data 1.0
73 mass_mixing_ratio_of_liquid_cloud_water_in_air... qlcs kg kg-1 time, layer qlcs 1.0
74 mass_mixing_ratio_of_rain_water_in_air_stratiform qlrs kg kg-1 time, layer qlrs 1.0
75 mass_mixing_ratio_of_ice_cloud_in_air_stratiform qics kg kg-1 time, layer missing data 1.0
76 mass_mixing_ratio_of_ice_precipitation_in_air_... qips kg kg-1 time, layer missing data 1.0
77 mass_mixing_ratio_of_liquid_cloud_water_in_air... qlcc kg kg-1 time, layer missing data 1.0
78 mass_mixing_ratio_of_rain_water_in_air_convective qlrc kg kg-1 time, layer missing data 1.0
79 mass_mixing_ratio_of_ice_cloud_in_air_convective qicc kg kg-1 time, layer missing data 1.0
80 mass_mixing_ratio_of_ice_precipitation_in_air_... qipc kg kg-1 time, layer missing data 1.0
81 number_of_liquid_cloud_droplets_in_air_stratiform nlcs kg-1 time, layer nlcs 1.0
82 number_of_rain_drops_in_air_stratiform nlrs kg-1 time, layer nlrs 1.0
83 number_of_ice_cloud_crystals_in_air_stratiform nics kg-1 time, layer missing data 1.0
84 number_of_ice_precipitation_crystals_in_air_st... nips kg-1 time, layer missing data 1.0
85 effective_radius_of_liquid_cloud_droplets_conv... relcc m time, layer missing data 1.0
86 effective_radius_of_rain_convective relrc m time, layer missing data 1.0
87 effective_radius_of_ice_cloud_convective reicc m time, layer missing data 1.0
88 effective_radius_of_ice_precipitation_convective reipc m time, layer missing data 1.0
89 area_fraction_of_liquid_cloud_stratiform flcs 1 time, layer flcs 1.0
90 area_fraction_of_rain_stratiform flrs 1 time, layer flrs 1.0
91 area_fraction_of_ice_cloud_stratiform fics 1 time, layer missing data 1.0
92 area_fraction_of_ice_precipitation_stratiform fips 1 time, layer missing data 1.0
93 area_fraction_of_liquid_cloud_convective flcc 1 time, layer missing data 1.0
94 area_fraction_of_rain_convective flrc 1 time, layer missing data 1.0
95 area_fraction_of_ice_cloud_convective ficc 1 time, layer missing data 1.0
96 area_fraction_of_ice_precipitation_convective fipc 1 time, layer missing data 1.0
97 mass_weighted_fall_speed_of_liquid_cloud_water... vmlcs m s-1 time, layer vmlcs 1.0
98 mass_weighted_fall_speed_of_rain_stratiform vmlrs m s-1 time, layer vmlrs 1.0
99 mass_weighted_fall_speed_of_ice_cloud_stratiform vmics m s-1 time, layer missing data 1.0
100 mass_weighted_fall_speed_of_ice_precipitation_... vmips m s-1 time, layer missing data 1.0
101 mass_weighted_fall_speed_of_liquid_cloud_water... vmlcc m s-1 time, layer missing data 1.0
102 mass_weighted_fall_speed_of_rain_convective vmlrc m s-1 time, layer missing data 1.0
103 mass_weighted_fall_speed_of_cloud_ice_crystals... vmicc m s-1 time, layer missing data 1.0
104 mass_weighted_fall_speed_of_ice_precipitation_... vmipc m s-1 time, layer missing data 1.0

Create DEPHY output file#

Write a single file to contain all domain-mean scalar and profile outputs. This code expects the write directory to be pre-existing (already created by the user). In the case that this output will be committed to the comble-mip GitHub repository, see above “Specify directory locations”.

# create DEPHY output file
dephy_filename = my_gitdir + my_output_filename
if os.path.exists(dephy_filename):
    os.remove(dephy_filename)
    print('The file ' + dephy_filename + ' has been deleted successfully')    
dephy_file = Dataset(dephy_filename,mode='w',format='NETCDF3_CLASSIC')
start_date = '2020-03-12T22:00:00Z'

# create global attributes
dephy_file.title='E3SMv2.1 SCM results for COMBLE-MIP case: fixed stratiform Nd without ice'
dephy_file.reference='https://github.com/ARM-Development/comble-mip'
dephy_file.authors='Meng Zhang (zhang55@llnl.gov), Xue Zheng (zheng7@llnl.gov), Peter Bogenschutz (bogenschutz1@llnl.gov), Yunyan Zhang (zhang25@llnl.gov), Shaocheng Xie (xie2@llnl.gov)'
dephy_file.source=input_filename
dephy_file.version=dt.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
dephy_file.format_version='DEPHY SCM format version 1.6'
dephy_file.script='convert_E3SMv2_SCM_output_to_dephy_format.ipynb'
dephy_file.startDate=start_date
dephy_file.force_geo=1
dephy_file.surfaceType='ocean'
dephy_file.surfaceForcing='ts'
dephy_file.lat='74.5 deg N'
dephy_file.dp='see pressure variable'
dephy_file.np=model_data.sizes['lev']
dephy_file.Note='E3SMv2.1 SCM run without nudging and the usage of geostrophic wind.'

# create dimensions
nt = model_data.dims['time']
time = dephy_file.createDimension('time', nt)
time = dephy_file.createVariable('time', np.float64, ('time',))
time.units = 'seconds since ' + dephy_file.startDate
time.long_name = 'time'
# find time step and build time in seconds
# time1 = dt.datetime.strptime(str(model_data['time'].data[0]),'%Y-%m-%d %H:%M:%S')
# time2 = dt.datetime.strptime(str(model_data['time'].data[1]),'%Y-%m-%d %H:%M:%S')
time1 = model_data['time'].data[0]
time2 = model_data['time'].data[1]
delta_t = (time2-time1).astype('timedelta64[s]') #.total_seconds()
time[:] = (np.arange(nt)+1.)*delta_t

nl = model_data.dims['lev']
pa = dephy_file.createDimension('layer', nl)
pa = dephy_file.createVariable('layer', np.float64, ('layer',))
pa.units = 'Pa'
pa.long_name = 'pressure'
pa[:] = model_data['lev'].data*100.

# create and fill variables
for index in vars_mean_list.index[2:]:
    std_name = vars_mean_list.standard_name.iat[index]
    print(std_name) # debug
    var_name = vars_mean_list.variable_id.iat[index]
    mod_name = vars_mean_list.model_name.iat[index]
    c_factor = vars_mean_list.conv_factor.iat[index]
    if vars_mean_list.dimensions.iat[index]=='time':
        new_sca = dephy_file.createVariable(var_name, np.float64, ('time'))
        new_sca.units = vars_mean_list.units.iat[index]
        new_sca.long_name = std_name
        if vars_mean_list.model_name.iat[index]!='missing data':
            new_sca[:] = model_data[mod_name].data*c_factor
    if vars_mean_list.dimensions.iat[index]=='time, layer':
        new_snd = dephy_file.createVariable(var_name, np.float64, ('time','layer'))
        new_snd.units = vars_mean_list.units.iat[index]
        new_snd.long_name = std_name
        if vars_mean_list.model_name.iat[index]!='missing data': 
            new_snd[:] = model_data[mod_name].data*c_factor

print(dephy_file)
dephy_file.close()

The file /home/mzhang1/Data_test/COMBLE-MIP-SCM/gitdir/E3SMv2-Phys_FixN_noice_def_z0_alt_no_ugvg.nc has been deleted successfully
air_pressure
layer_top_pressure
surface_pressure
surface_temperature
surface_friction_velocity
surface_roughness_length_for_momentum_in_air
surface_roughness_length_for_heat_in_air
surface_roughness_length_for_humidity_in_air
surface_upward_sensible_heat_flux
surface_upward_latent_heat_flux
obukhov_length
pbl_height
inversion_height
atmosphere_mass_content_of_liquid_cloud_water
atmosphere_mass_content_of_rain_water
atmosphere_mass_content_of_ice_water
area_fraction_cover_of_hydrometeors
area_fraction_cover_of_liquid_cloud
area_fraction_cover_of_convective_hydrometeors
optical_depth
optical_depth_of_liquid_cloud
precipitation_flux_at_surface
precipitation_flux_of_ice_at_surface
toa_outgoing_longwave_flux
surface_downwelling_longwave_flux
surface_upwelling_longwave_flux
surface_sea_spray_number_flux
height
eastward_wind
northward_wind
air_dry_density
air_temperature
water_vapor_mixing_ratio
relative_humidity
relative_humidity_over_ice
air_potential_temperature
mass_mixing_ratio_of_cloud_liquid_water_in_air
mass_mixing_ratio_of_rain_water_in_air
mass_mixing_ratio_of_ice_water_in_air
area_fraction_of_hydrometeors
area_fraction_of_liquid_cloud
area_fraction_of_convective_hydrometeors
precipitation_flux_in_air
precipitation_flux_in_air_in_ice_phase
specific_turbulent_kinetic_energy
dissipation_rate_of_turbulent_kinetic_energy
zonal_momentum_flux
meridional_momentum_flux
variance_of_upward_air_velocity
vertical_flux_potential_temperature
vertical_flux_liquid_ice_water_potential_temperature
vertical_flux_water_vapor
vertical_flux_total_water
convection_updraft_mass_flux
convection_downdraft_mass_flux
downwelling_longwave_flux_in_air
upwelling_longwave_flux_in_air
tendency_of_air_potential_temperature_due_to_radiative_heating
tendency_of_air_potential_temperature_due_to_microphysics
tendency_of_air_potential_temperature_due_to_mixing
tendency_of_water_vapor_mixing_ratio_due_to_microphysics
tendency_of_water_vapor_mixing_ratio_due_to_mixing
number_of_total_aerosol_mode1
number_of_total_aerosol_mode2
number_of_total_aerosol_mode3
tendency_of_aerosol_number_due_to_warm_microphysics
tendency_of_aerosol_number_due_to_cold_microphysics
tendency_of_aerosol_number_due_to_mixing
tendency_of_ice_number_due_to_heterogeneous_freezing
tendency_of_ice_number_due_to_secondary_ice_production
tendency_of_ice_number_due_to_homogeneous_freezing
mass_mixing_ratio_of_liquid_cloud_water_in_air_stratiform
mass_mixing_ratio_of_rain_water_in_air_stratiform
mass_mixing_ratio_of_ice_cloud_in_air_stratiform
mass_mixing_ratio_of_ice_precipitation_in_air_stratiform
mass_mixing_ratio_of_liquid_cloud_water_in_air_convective
mass_mixing_ratio_of_rain_water_in_air_convective
mass_mixing_ratio_of_ice_cloud_in_air_convective
mass_mixing_ratio_of_ice_precipitation_in_air_convective
number_of_liquid_cloud_droplets_in_air_stratiform
number_of_rain_drops_in_air_stratiform
number_of_ice_cloud_crystals_in_air_stratiform
number_of_ice_precipitation_crystals_in_air_stratiform
effective_radius_of_liquid_cloud_droplets_convective
effective_radius_of_rain_convective
effective_radius_of_ice_cloud_convective
effective_radius_of_ice_precipitation_convective
area_fraction_of_liquid_cloud_stratiform
area_fraction_of_rain_stratiform
area_fraction_of_ice_cloud_stratiform
area_fraction_of_ice_precipitation_stratiform
area_fraction_of_liquid_cloud_convective
area_fraction_of_rain_convective
area_fraction_of_ice_cloud_convective
area_fraction_of_ice_precipitation_convective
mass_weighted_fall_speed_of_liquid_cloud_water_stratiform
mass_weighted_fall_speed_of_rain_stratiform
mass_weighted_fall_speed_of_ice_cloud_stratiform
mass_weighted_fall_speed_of_ice_precipitation_stratiform
mass_weighted_fall_speed_of_liquid_cloud_water_convective
mass_weighted_fall_speed_of_rain_convective
mass_weighted_fall_speed_of_cloud_ice_crystals_convective
mass_weighted_fall_speed_of_ice_precipitation_convective
<class 'netCDF4._netCDF4.Dataset'>
root group (NETCDF3_CLASSIC data model, file format NETCDF3):
    title: E3SMv2.1 SCM results for COMBLE-MIP case: fixed stratiform Nd without ice
    reference: https://github.com/ARM-Development/comble-mip
    authors: Meng Zhang (zhang55@llnl.gov), Xue Zheng (zheng7@llnl.gov), Peter Bogenschutz (bogenschutz1@llnl.gov), Yunyan Zhang (zhang25@llnl.gov), Shaocheng Xie (xie2@llnl.gov)
    source: /home/mzhang1/Data_test/COMBLE-MIP-SCM/COMBLE-MIP_v2_4_e3smv2_1_FixN_noice.eam.h0.2020-03-12-79200.nc
    version: 2024-06-21 21:25:57
    format_version: DEPHY SCM format version 1.6
    script: convert_E3SMv2_SCM_output_to_dephy_format.ipynb
    startDate: 2020-03-12T22:00:00Z
    force_geo: 1
    surfaceType: ocean
    surfaceForcing: ts
    lat: 74.5 deg N
    dp: see pressure variable
    np: 72
    Note: E3SMv2.1 SCM run without nudging and the usage of geostrophic wind.
    dimensions(sizes): time(41), layer(72)
    variables(dimensions): float64 time(time), float64 layer(layer), float64 pa(time, layer), float64 pe(time, layer), float64 ps(time), float64 ts(time), float64 ustar(time), float64 z0(time), float64 z0h(time), float64 z0q(time), float64 hfss(time), float64 hfls(time), float64 ol(time), float64 pblh(time), float64 zi(time), float64 lwpc(time), float64 lwpr(time), float64 iwp(time), float64 cf(time), float64 cflc(time), float64 cfc(time), float64 od(time), float64 odlc(time), float64 pr(time), float64 pri(time), float64 rlut(time), float64 rlds(time), float64 rlus(time), float64 ssaf(time), float64 zf(time, layer), float64 ua(time, layer), float64 va(time, layer), float64 rhoa(time, layer), float64 ta(time, layer), float64 qv(time, layer), float64 hur(time, layer), float64 huri(time, layer), float64 theta(time, layer), float64 qlc(time, layer), float64 qlr(time, layer), float64 qi(time, layer), float64 fh(time, layer), float64 flc(time, layer), float64 fc(time, layer), float64 prf(time, layer), float64 prfi(time, layer), float64 tke(time, layer), float64 eps(time, layer), float64 uw(time, layer), float64 vw(time, layer), float64 w2(time, layer), float64 wth(time, layer), float64 vf_thli(time, layer), float64 wqv(time, layer), float64 vf_qt(time, layer), float64 cmfu(time, layer), float64 cmfd(time, layer), float64 rld(time, layer), float64 rlu(time, layer), float64 dth_rad(time, layer), float64 dth_micro(time, layer), float64 dth_turb(time, layer), float64 dq_micro(time, layer), float64 dq_turb(time, layer), float64 na1(time, layer), float64 na2(time, layer), float64 na3(time, layer), float64 dna_micro_warm(time, layer), float64 dna_micro_cold(time, layer), float64 dna_turb(time, layer), float64 dni_het(time, layer), float64 dni_sip(time, layer), float64 dni_hom(time, layer), float64 qlcs(time, layer), float64 qlrs(time, layer), float64 qics(time, layer), float64 qips(time, layer), float64 qlcc(time, layer), float64 qlrc(time, layer), float64 qicc(time, layer), float64 qipc(time, layer), float64 nlcs(time, layer), float64 nlrs(time, layer), float64 nics(time, layer), float64 nips(time, layer), float64 relcc(time, layer), float64 relrc(time, layer), float64 reicc(time, layer), float64 reipc(time, layer), float64 flcs(time, layer), float64 flrs(time, layer), float64 fics(time, layer), float64 fips(time, layer), float64 flcc(time, layer), float64 flrc(time, layer), float64 ficc(time, layer), float64 fipc(time, layer), float64 vmlcs(time, layer), float64 vmlrs(time, layer), float64 vmics(time, layer), float64 vmips(time, layer), float64 vmlcc(time, layer), float64 vmlrc(time, layer), float64 vmicc(time, layer), float64 vmipc(time, layer)
    groups:

Check output file#

dephy_check = xr.open_dataset(dephy_filename)
print('File name ',dephy_filename)
dephy_check

File name  /home/mzhang1/Data_test/COMBLE-MIP-SCM/gitdir/E3SMv2-Phys_FixN_noice_def_z0_alt_no_ugvg.nc

<xarray.Dataset>
Dimensions:         (time: 41, layer: 72)
Coordinates:
  * time            (time) datetime64[ns] 2020-03-12T22:30:00 ... 2020-03-13T...
  * layer           (layer) float64 12.38 18.28 26.99 ... 9.938e+04 9.985e+04
Data variables: (12/103)
    pa              (time, layer) float64 ...
    pe              (time, layer) float64 ...
    ps              (time) float64 ...
    ts              (time) float64 ...
    ustar           (time) float64 ...
    z0              (time) float64 ...
    ...              ...
    vmics           (time, layer) float64 ...
    vmips           (time, layer) float64 ...
    vmlcc           (time, layer) float64 ...
    vmlrc           (time, layer) float64 ...
    vmicc           (time, layer) float64 ...
    vmipc           (time, layer) float64 ...
Attributes: (12/15)
    title:           E3SMv2.1 SCM results for COMBLE-MIP case: fixed stratifo...
    reference:       https://github.com/ARM-Development/comble-mip
    authors:         Meng Zhang (zhang55@llnl.gov), Xue Zheng (zheng7@llnl.go...
    source:          /home/mzhang1/Data_test/COMBLE-MIP-SCM/COMBLE-MIP_v2_4_e...
    version:         2024-06-21 21:25:57
    format_version:  DEPHY SCM format version 1.6
    ...              ...
    surfaceType:     ocean
    surfaceForcing:  ts
    lat:             74.5 deg N
    dp:              see pressure variable
    np:              72
    Note:            E3SMv2.1 SCM run without nudging and the usage of geostr...