![\"NEON-ARM\"](\"https://arm-doe.github.io/ACT/_images/sphx_glr_plot_neon_001.png\")
\n",
" ![\"SONDE\"](\"https://arm-doe.github.io/ACT/_images/sphx_glr_plot_enhanced_skewt_001.png\")
\n",
" ![](\"https://raw.githubusercontent.com/ProjectPythiaCookbooks/radar-cookbook/main/thumbnail.png\")
"
]
},
{
"cell_type": "markdown",
"id": "6ce8e709-11fb-4775-bc35-53e3120b0d22",
"metadata": {},
"source": [
"# ACT Basics"
]
},
{
"cell_type": "markdown",
"id": "fdddd23b-cf90-44ee-97c7-cd91494f1392",
"metadata": {},
"source": [
"## Overview\n",
"The ARM COMBLE campaign collected a lot of very interesting data in Norway from December 1 2019 - May 31 2020. Here we look at a case study from March 9 to March 16, 2020. This notebook will give an introduction to basic features in ACT, using some relevant datastreams from this event\n",
"\n",
"1. Intro to ACT\n",
"1. Downloading and Reading in Data\n",
"1. Quality Controlling Data\n",
"1. Aerosol Instrument Overview\n",
"1. Visualizing Data\n",
"1. Additional Features in ACT"
]
},
{
"cell_type": "markdown",
"id": "3de97bb8-5960-4ad8-9d43-cce9d2885f20",
"metadata": {
"tags": []
},
"source": [
"## Prerequisites\n",
"This notebook will rely heavily on Python and the [Atmospheric data Community Toolkit (ACT)](https://github.com/ARM-DOE/ACT). Don't worry if you don't have experience with either, this notebook will walk you though what you need to know.\n",
"\n",
"You will also need an account and token to download data using the ARM Live webservice. Navigate to the [webservice information page](https://adc.arm.gov/armlive/) and log in to get your token. Your account username will be your ARM username.\n",
"\n",
"| Concepts | Importance | Notes |\n",
"| --- | --- | --- |\n",
"| [ACT](https://github.com/ARM-DOE/ACT) | Helpful | |\n",
"\n",
"- **Time to learn**: 60 Minutes\n",
"- **System requirements**:\n",
" - Python 3.11 or latest\n",
" - ACT v1.5.0 or latest\n",
" - numpy\n",
" - xarray\n",
" - matplotlib"
]
},
{
"cell_type": "markdown",
"id": "e02cda1c-add4-42b6-9a62-5f6d575f4560",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"id": "e048ef19-fa37-4673-b567-341ab53bb426",
"metadata": {},
"source": [
"## Intro to ACT\n",
" \n",
"The [Atmospheric data Community Toolkit (ACT)](https://github.com/ARM-DOE/ACT) is an open-source Python toolkit for exploring and analyzing atmospheric time-series datasets. Examples can be found in the [ACT Example Gallery](https://arm-doe.github.io/ACT/source/auto_examples/index.html). The toolkit has modules for many different parts of the scientific process, including:\n",
"\n",
"| \n", " \n", "#### Data Discovery (act.discovery)\n", "The [discovery](https://arm-doe.github.io/ACT/API/generated/act.discovery.html) module houses functions to download or access data from different groups. Currently it includes function to get data for ARM, NOAA, EPA, NEON, and more!\n", "\n", "#### Input/Output (act.io)\n", "[io](https://arm-doe.github.io/ACT/API/generated/act.io.html) contains functions for reading and writing data from various sources and formats.\n", "\n", "#### Visualization (act.plotting)\n", "[plotting](https://arm-doe.github.io/ACT/API/generated/act.plotting.html) contains various routines, built on matplotlib, to help visualize and explore data. These include \n", "1. Time-series plots\n", "1. Distribution plots like histograms and heatmaps\n", "1. Geographic plots for moving systems like radiosondes or aircraft\n", "1. Skew-T plots for radiosonde data, built off MetPy\n", "1. Wind rose plots for wind and data roses\n", "1. Cross-section plots for working with 3-dimensional data\n", "\n", "#### Corrections (act.corrections)\n", "[corrections](https://arm-doe.github.io/ACT/API/generated/act.corrections.html) apply different corrections to data based on need. A majority of the existing corrections are for lidar data.\n", "\n", "#### Quality Control (act.qc)\n", "The [qc](https://arm-doe.github.io/ACT/API/generated/act.qc.html) module has a lot of functions for working with quality control information, apply new tests, or filtering data based on existing tests. We will explore some of that functionality in this notebook.\n", "\n", "#### Retrievals (act.retrievals)\n", "There are many cases in which some additional calculations are necessary to get more value from the instrument data. The [retrievals](https://arm-doe.github.io/ACT/API/generated/act.retrievals.html) module houses some functions for performing these advanced calculations. \n", "\n", "#### Utilities (act.utils)\n", "The [utils](https://arm-doe.github.io/ACT/API/generated/act.utils.html) module has a lot of general utilities to help with the data. Some of these include adding in a solar variable to indicate day/night (useful in filtering data), unit conversions, decoding WMO weather codes, performing weighted averaging, etc...\n", " | \n", "\n",
" \n",
" \n",
" | \n",
"
<xarray.Dataset> Size: 277MB\n",
"Dimensions: (time: 8640, range_bins: 3990,\n",
" num_darkcount_corr: 3990,\n",
" num_deadtime_corr: 23,\n",
" num_overlap_corr: 332)\n",
"Coordinates:\n",
" * time (time) datetime64[ns] 69kB 2020-03-12T00...\n",
" * range_bins (range_bins) float32 16kB 0.01499 ... 59.81\n",
"Dimensions without coordinates: num_darkcount_corr, num_deadtime_corr,\n",
" num_overlap_corr\n",
"Data variables: (12/46)\n",
" base_time datetime64[ns] 8B 2020-03-12\n",
" time_offset (time) datetime64[ns] 69kB 2020-03-12T00...\n",
" num_channels int32 4B ...\n",
" num_bins int32 4B ...\n",
" range (range_bins) float32 16kB dask.array<chunksize=(3990,), meta=np.ndarray>\n",
" height (range_bins) float32 16kB dask.array<chunksize=(3990,), meta=np.ndarray>\n",
" ... ...\n",
" qc_signal_return_co_pol (time) int32 35kB dask.array<chunksize=(8640,), meta=np.ndarray>\n",
" first_data_bin (time) int32 35kB dask.array<chunksize=(8640,), meta=np.ndarray>\n",
" laser_fire_bin (time) int32 35kB dask.array<chunksize=(8640,), meta=np.ndarray>\n",
" lat float32 4B ...\n",
" lon float32 4B ...\n",
" alt float32 4B ...\n",
"Attributes: (12/33)\n",
" command_line: mplpolfs_ingest -s anx -f M1\n",
" Conventions: ARM-1.2\n",
" process_version: ingest-mplpolfs-1.23-0.el7\n",
" ingest_software: ingest-mplpolfs-1.23-0.el7\n",
" dod_version: mplpolfs-b1-3.0\n",
" site_id: anx\n",
" ... ...\n",
" doi: 10.5439/1320657\n",
" history: created by user dsmgr on machine flint at 20...\n",
" _file_dates: ['20200312']\n",
" _file_times: ['000005']\n",
" _datastream: anxmplpolfsM1.b1\n",
" _arm_standards_flag: 1<xarray.DataArray 'qc_signal_return_co_pol' (time: 8640, range_bins: 2183)> Size: 75MB\n",
"array([[0, 0, 0, ..., 0, 0, 0],\n",
" [0, 0, 0, ..., 0, 0, 0],\n",
" [0, 0, 0, ..., 0, 0, 0],\n",
" ...,\n",
" [0, 0, 0, ..., 0, 0, 0],\n",
" [0, 0, 0, ..., 0, 0, 0],\n",
" [0, 0, 0, ..., 0, 0, 0]], dtype=uint32)\n",
"Coordinates:\n",
" * time (time) datetime64[ns] 69kB 2020-03-12T00:00:05 ... 2020-03-12...\n",
" * range_bins (range_bins) float32 9kB 27.1 27.12 27.13 ... 59.78 59.79 59.81\n",
"Attributes:\n",
" long_name: Quality check results on field: Attenuated backscatter...\n",
" units: 1\n",
" flag_masks: [1, 2, 4, 8, 16]\n",
" flag_meanings: ['Not used', 'Not used', 'Not used', 'Not used', 'The ...\n",
" flag_assessments: ['Bad', 'Bad', 'Bad', 'Bad', 'Bad']\n",
" standard_name: quality_flag\n",
" missing_value: 4294957297| \n", "\n", "### Single Particle Soot Photometer (SP2)\n", "The single-particle soot photometer (SP2) measures the soot (black carbon) mass of individual aerosol particles by laser-induced incandescence down to concentrations as low as ng/m^3. [Learn more](https://arm.gov/capabilities/instruments/sp2)\n", "\n", "### Aerodynamic Particle Sizer (APS)\n", "The aerodynamic particle sizer (APS) is a particle size spectrometer that measures both the particle aerodynamic diameter based on particle time of flight and optical diameter based on scattered light intensity. The APS provides the number size distribution for particles with aerodynamic diameters from 0.5 to 20 micrometers and with optical diameters from 0.3 to 20 micrometers. [Learn more](https://arm.gov/capabilities/instruments/aps)\n", "\n", "### Aerosol Chemical Speciation Monitor (ACSM)\n", "The aerosol chemical speciation monitor is a thermal vaporization, electron impact, ionization mass spectrometer that measures bulk chemical composition of the rapidly evaporating component of sub-micron aerosol particles in real time. Standard measurements include mass concentrations of organics, sulfate, nitrate, ammonium, and chloride. [Learn more](https://arm.gov/capabilities/instruments/acsm)\n", " | \n", "\n",
" ![](\"../images/aerosol_sizing.png\") \n",
" | \n",
"
<xarray.Dataset> Size: 377kB\n",
"Dimensions: (time: 108, bound: 2,\n",
" diameter_mobility: 192, droplet_size: 20,\n",
" setpoint: 7)\n",
"Coordinates:\n",
" * time (time) datetime64[ns] 864B 2020-03-09T00...\n",
" * diameter_mobility (diameter_mobility) float32 768B 1.02 .....\n",
" * droplet_size (droplet_size) float32 80B 0.0 1.0 ... 19.0\n",
" * setpoint (setpoint) float32 28B 0.0 0.1 ... 0.8 1.0\n",
"Dimensions without coordinates: bound\n",
"Data variables: (12/22)\n",
" base_time (time) datetime64[ns] 864B 2020-03-09 .....\n",
" time_offset (time) datetime64[ns] 864B 2020-03-09T00...\n",
" time_bounds (time, bound) object 2kB dask.array<chunksize=(14, 2), meta=np.ndarray>\n",
" diameter_mobility_bounds (time, diameter_mobility, bound) float32 166kB dask.array<chunksize=(14, 192, 2), meta=np.ndarray>\n",
" droplet_size_bounds (time, droplet_size, bound) float32 17kB dask.array<chunksize=(14, 20, 2), meta=np.ndarray>\n",
" kappa (time) float32 432B dask.array<chunksize=(18,), meta=np.ndarray>\n",
" ... ...\n",
" qc_N_CCN_dN (time, droplet_size) int32 9kB dask.array<chunksize=(14, 20), meta=np.ndarray>\n",
" dN_dlogDp (time, diameter_mobility) float32 83kB dask.array<chunksize=(18, 192), meta=np.ndarray>\n",
" qc_dN_dlogDp (time, diameter_mobility) int32 83kB dask.array<chunksize=(14, 192), meta=np.ndarray>\n",
" lat (time) float32 432B 69.14 69.14 ... 69.14\n",
" lon (time) float32 432B 15.68 15.68 ... 15.68\n",
" alt (time) float32 432B 2.0 2.0 2.0 ... 2.0 2.0\n",
"Attributes: (12/17)\n",
" command_line: aosccnsmpskappa -n aosccnsmpskappa -s anx -f M1 -b...\n",
" Conventions: ARM-1.3\n",
" process_version: aosccnsmpskappa-1.2.0\n",
" dod_version: aosccnsmpskappa-c1-1.2\n",
" input_datastreams: anxaosccn2colaavgM1.b1 : 2.1 : 20200309.003204\\nan...\n",
" site_id: anx\n",
" ... ...\n",
" doi: 10.5439/1729907\n",
" history: created by user levin on machine prod-proc5.adc.ar...\n",
" _file_dates: ['20200309', '20200310', '20200311', '20200312', '...\n",
" _file_times: ['003204', '002602', '022138', '013206', '003849',...\n",
" _datastream: anxaosccnsmpskappaM1.c1\n",
" _arm_standards_flag: 1