Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

ACT Basics: An Overview of the ARM Deployment at Cape Grim

Authors
Affiliations
Argonne National Laboratory
Argonne National Laboratory
The University of Wollongong
Queensland University of Technology
The University of Melbourne
Leibniz Institute for. Tropospheric Research
The University of Melbourne
Monash University
The University of Michigan
The University of Michigan
The University of Southern Queensland
The Commonwealth Scientific and Industrial Research Organisation
The Commonwealth Scientific and Industrial Research Organisation
The University of Wollongong
The Commonwealth Scientific and Industrial Research Organisation
Australia's Nuclear Science and Technology Organisation
The Commonwealth Scientific and Industrial Research Organisation

ACT Basics: An Overview of the ARM Deployment at Cape Grim

Overview

The ARM CAPE-k campaign is collecting interesting data in Tasmania, Australia from April 15 2024 - September 15 2025. Here we look at a case study from November 7 to November 14, 2024. This notebook will give an introduction to basic features in ACT, using some relevant datastreams from this event

  1. Intro to ACT

  2. Downloading and Reading in Data

  3. Quality Controlling Data

  4. Aerosol Instrument Overview

  5. Visualizing Data

  6. Additional Features in ACT

Prerequisites

This notebook will rely heavily on Python and the Atmospheric data Community Toolkit (ACT). Don’t worry if you don’t have experience with either, this notebook will walk you though what you need to know.

You will also need an account and token to download data using the ARM Live webservice. Navigate to the webservice information page and log in to get your token. Your account username will be your ARM username.

ConceptsImportanceNotes
ACTHelpful
  • Time to learn: 60 Minutes

  • System requirements:

    • Python 3.11 or latest

    • ACT v1.5.0 or latest

    • numpy

    • xarray

    • matplotlib


Intro to ACT

The Atmospheric data Community Toolkit (ACT) is an open-source Python toolkit for exploring and analyzing atmospheric time-series datasets. Examples can be found in the ACT Example Gallery. The toolkit has modules for many different parts of the scientific process, including:

Enhanced Skew-T plot from ARM’s Southern Great Plains Site (SGP)

Data Discovery (act.discovery)

The discovery module houses functions to download or access data from different groups. Currently it includes function to get data for ARM, NOAA, EPA, NEON, and more!

Input/Output (act.io)

io contains functions for reading and writing data from various sources and formats.

Visualization (act.plotting)

plotting contains various routines, built on matplotlib, to help visualize and explore data. These include

  1. Time-series plots

  2. Distribution plots like histograms and heatmaps

  3. Geographic plots for moving systems like radiosondes or aircraft

  4. Skew-T plots for radiosonde data, built off MetPy

  5. Wind rose plots for wind and data roses

  6. Cross-section plots for working with 3-dimensional data

Corrections (act.corrections)

corrections apply different corrections to data based on need. A majority of the existing corrections are for lidar data.

Quality Control (act.qc)

The qc module has a lot of functions for working with quality control information, apply new tests, or filtering data based on existing tests. We will explore some of that functionality in this notebook.

Retrievals (act.retrievals)

There are many cases in which some additional calculations are necessary to get more value from the instrument data. The retrievals module houses some functions for performing these advanced calculations.

Utilities (act.utils)

The utils module has a lot of general utilities to help with the data. Some of these include adding in a solar variable to indicate day/night (useful in filtering data), unit conversions, decoding WMO weather codes, performing weighted averaging, etc...

NEON-ARM

ARM and NEON data from Utquivaik, AK

SONDE

Imports

Let’s get started with some data! But first, we need to import some libraries.

Downloading and Reading ARM’s NetCDF Data

ARM’s standard file format is NetCDF (network Common Data Form) which makes it very easy to work with in Python! ARM data are available through a data portal called Data Discovery or through a webservice. If you didn’t get your username and token earlier, please go back and see the Prerequisites!

Let’s download some of the MPL data first but let’s just start with one day.

[DOWNLOADING] kcgmplpolfsM1.b1.20241110.000001.nc

If you use these data to prepare a publication, please cite:

Muradyan, P., Cromwell, E., Koontz, A., Coulter, R., Flynn, C., Ermold, B., &
OBrien, J. Micropulse Lidar (MPLPOLFS), 2024-11-10 to 2024-11-11, ARM Mobile
Facility (KCG), kennaook ⁄ Cape Grim, Tasmania, Australia; AMF2 (main site for
CAPE-k) (M1). Atmospheric Radiation Measurement (ARM) User Facility.
https://doi.org/10.5439/1320657

Loading...

Quality Controlling Data

ARM has multiple methods that it uses to communicate data quality information out to the users. One of these methods is through “embedded QC” variables. These are variables within the file that have information on automated tests that have been applied. Many times, they include Min, Max, and Delta tests but as is the case with the AOS instruments, there can be more complicated tests that are applied.

The results from all these different tests are stored in a single variable using bit-packed QC. We won’t get into the full details here, but it’s a way to communicate the results of multiple tests in a single integer value by utilizing binary and bits! You can learn more about bit-packed QC here but ACT also has many of the tools for working with ARM QC.

Other Sources of Quality Control

ARM also communicates problems with the data quality through Data Quality Reports (DQR). These reports are normally submitted by the instrument mentor when there’s been a problem with the instrument. The categories include:

  • Data Quality Report Categories

    • Missing: Data are not available or set to -9999

    • Suspect: The data are not fully incorrect but there are problems that increases the uncertainty of the values. Data should be used with caution.

    • Bad: The data are incorrect and should not be used.

    • Note: Data notes are a way to communicate information that would be useful to the end user but does not rise to the level of suspect or bad data

Examples of ACT QC functionality

Additionally, data quality information can be found in the Instrument Handbooks, which are included on most instrument pages. Here is an example of the MPL handbook.

<Figure size 1000x500 with 2 Axes>

Filtering data

It’s easy to filter out data failing tests with ACT. This will show you how to filter data by test or by assessment.

{'long_name': 'Attenuated backscatter, copol', 'units': '10 * log10(count/us)', 'description': 'This field literally contains the counts detected by the detector for each range bin. No corrections of any kind have been applied to this field. In order to make proper use of this data, one should correct for detector non-linearity, subtract background counts, apply a range-squared multiplication, and optimally correct for optical overlap and collimation artifacts.', 'ancillary_variables': 'qc_signal_return_co_pol', 'history': 'act.qc.datafilter: Not used\n act.qc.datafilter: Not used\n act.qc.datafilter: Not used\n act.qc.datafilter: Not used'}
<Figure size 1500x1000 with 4 Axes>

ARM Data Quality Reports (DQR)!

ARM’s DQRs can be easily pulled in and added to the QC variables using ACT. We can do that with the below one line command. However, for this case, there won’t be any DQRs on the data but let’s visualize it just in case! Check out the ACT QC Examples for more use cases!

Loading...

Aerosol Instrument Overview

Single Particle Soot Photometer (SP2)

The single-particle soot photometer (SP2) measures the soot (black carbon) mass of individual aerosol particles by laser-induced incandescence down to concentrations as low as ng/m^3. Learn more

Aerodynamic Particle Sizer (APS)

The aerodynamic particle sizer (APS) is a particle size spectrometer that measures both the particle aerodynamic diameter based on particle time of flight and optical diameter based on scattered light intensity. The APS provides the number size distribution for particles with aerodynamic diameters from 0.5 to 20 micrometers and with optical diameters from 0.3 to 20 micrometers. Learn more

Aerosol Chemical Speciation Monitor (ACSM)

The aerosol chemical speciation monitor is a thermal vaporization, electron impact, ionization mass spectrometer that measures bulk chemical composition of the rapidly evaporating component of sub-micron aerosol particles in real time. Standard measurements include mass concentrations of organics, sulfate, nitrate, ammonium, and chloride. Learn more

ARM Aerosol Instrumentation Particle Size Ranges

Downloading and QCing the Aerosol Data

Let’s start pulling these data together into the same plots so we can see what’s going on.

[DOWNLOADING] kcgaosapsS3.b1.20241113.000000.nc
[DOWNLOADING] kcgaosapsS3.b1.20241107.000000.nc
[DOWNLOADING] kcgaosapsS3.b1.20241111.000000.nc
[DOWNLOADING] kcgaosapsS3.b1.20241108.000000.nc
[DOWNLOADING] kcgaosapsS3.b1.20241109.000000.nc
[DOWNLOADING] kcgaosapsS3.b1.20241110.000000.nc
[DOWNLOADING] kcgaosapsS3.b1.20241112.000000.nc

If you use these data to prepare a publication, please cite:

Singh, A., Oliveira, D., Koontz, A., Sedlacek, A., & Kuang, C. Aerodynamic
Particle Sizer (AOSAPS), 2024-11-07 to 2024-11-14, ARM Mobile Facility (KCG),
kennaook  ⁄  Cape Grim, Tasmania, Australia; Supplementary Facility 3 (S3).
Atmospheric Radiation Measurement (ARM) User Facility.
https://doi.org/10.5439/1407135

[DOWNLOADING] kcgaosmetS2.a1.20241111.000000.nc
[DOWNLOADING] kcgaosmetS2.a1.20241107.000000.nc
[DOWNLOADING] kcgaosmetS2.a1.20241109.000000.nc
[DOWNLOADING] kcgaosmetS2.a1.20241113.000000.nc
[DOWNLOADING] kcgaosmetS2.a1.20241110.000000.nc
[DOWNLOADING] kcgaosmetS2.a1.20241112.000000.nc
[DOWNLOADING] kcgaosmetS2.a1.20241108.000000.nc

If you use these data to prepare a publication, please cite:

Kyrouac, J., & Tuftedal, M. Meteorological Measurements associated with the
Aerosol Observing System (AOSMET), 2024-11-07 to 2024-11-14, ARM Mobile Facility
(KCG), kennaook ⁄ Cape Grim, Tasmania, Australia; Supplementary Facility 2 (S2).
Atmospheric Radiation Measurement (ARM) User Facility.
https://doi.org/10.5439/1984920

[DOWNLOADING] kcg30smplcmask1zwangM1.c1.20241111.000002.nc
[DOWNLOADING] kcg30smplcmask1zwangM1.c1.20241107.000008.nc
[DOWNLOADING] kcg30smplcmask1zwangM1.c1.20241110.000001.nc
[DOWNLOADING] kcg30smplcmask1zwangM1.c1.20241112.000003.nc
[DOWNLOADING] kcg30smplcmask1zwangM1.c1.20241113.000003.nc
[DOWNLOADING] kcg30smplcmask1zwangM1.c1.20241108.000009.nc
[DOWNLOADING] kcg30smplcmask1zwangM1.c1.20241109.000000.nc

If you use these data to prepare a publication, please cite:

Sivaraman, C., Flynn, D., Riihimaki, L., Comstock, J., & Zhang, D. Cloud mask
from Micropulse Lidar (30SMPLCMASK1ZWANG), 2024-11-07 to 2024-11-14, ARM Mobile
Facility (KCG), kennaook ⁄ Cape Grim, Tasmania, Australia; AMF2 (main site for
CAPE-k) (M1). Atmospheric Radiation Measurement (ARM) User Facility.
https://doi.org/10.5439/1508389

Loading...

Visualizing Data

We have all the datasets downloaded, let’s start to visualize them in different ways using ACT. If you ever need a place to start with how to visualize data using ACT, check out the ACT Plotting Examples

<Figure size 1000x1600 with 5 Axes>
<Figure size 640x480 with 1 Axes>

Data Rose Plots

These plots display the data on a windrose-like plot to visualize directional dependencies in the data.

<Figure size 640x480 with 1 Axes>

Checkout the area

The AMF is deployed in at the Cape Grim site. Check out the google map and see if this mapes sense!

Back to the visualizations!

Let’s get back to checking out the other visualization features in ACT!

Histograms

<Figure size 640x480 with 1 Axes>
<Figure size 1500x1500 with 24 Axes>

Scatter Plots and Heatmaps

Let’s plot up a comparison of the APS total concentration and the ACSM sulfates. Feel free to change the variables from the ACSM to experiment!

<Figure size 640x480 with 2 Axes>
<Figure size 1200x500 with 4 Axes>

Create a Visualization of the Number Concentration by Size

One of the other dimensions of the CCN dataset is by the size, in nanometers, ranging from 1 to 19 micrometers. To plot this, we can create new variables in the dataset, then create a loop to plot these.

<Figure size 1400x500 with 1 Axes>

Additional Features in ACT

If there’s time to explore more features or if you want to on your own time, these are some of the many additional features that you might find useful in ACT

Skew-T Plots

[DOWNLOADING] kcgsondewnpnM1.b1.20241108.113000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241108.053000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241109.233000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241112.113000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241109.173000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241113.113000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241109.053000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241109.113000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241110.113000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241112.053000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241110.053000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241107.113000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241107.233000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241108.173000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241111.233000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241113.233000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241110.173000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241111.173000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241111.053000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241110.233000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241108.233000.cdf
[DOWNLOADING] kcgsondewnpnM1.b1.20241111.113000.cdf

If you use these data to prepare a publication, please cite:

Keeler, E., Burk, K., & Kyrouac, J. Balloon-Borne Sounding System (SONDEWNPN),
2024-11-07 to 2024-11-14, ARM Mobile Facility (KCG), kennaook ⁄ Cape Grim,
Tasmania, Australia; AMF2 (main site for CAPE-k) (M1). Atmospheric Radiation
Measurement (ARM) User Facility. https://doi.org/10.5439/1595321

<Figure size 640x480 with 3 Axes>

Wind Roses

<Figure size 1000x800 with 1 Axes>
Loading...

Accumulating Precipitation

This example shows how to accumulate precipitation using the ACT utility and then overplot the PWD present weather codes

<Figure size 800x600 with 1 Axes>

Doppler Lidar Wind Retrievals

This will show you how you can process the doppler lidar PPI scans to produce wind profiles based on Newsom et al 2016.

[DOWNLOADING] kcgdlppiM1.b1.20241110.013523.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.054540.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.032530.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.042534.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.012522.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.050538.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.033531.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.021526.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.010521.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.034532.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.053540.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.014524.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.052539.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.043536.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.045537.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.055541.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.002516.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.024528.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.040534.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.011522.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.051539.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.035532.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.044536.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.000516.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.041534.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.005519.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.015525.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.022527.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.001516.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.031530.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.003517.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.004518.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.025529.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.030529.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.023527.cdf
[DOWNLOADING] kcgdlppiM1.b1.20241110.020526.cdf

If you use these data to prepare a publication, please cite:

Newsom, R., Shi, Y., & Krishnamurthy, R. Doppler Lidar (DLPPI), 2024-11-10 to
2024-11-10, ARM Mobile Facility (KCG), kennaook ⁄ Cape Grim, Tasmania,
Australia; AMF2 (main site for CAPE-k) (M1). Atmospheric Radiation Measurement
(ARM) User Facility. https://doi.org/10.5439/1025186

<Figure size 640x480 with 1 Axes>