PySP2

PySP2#

PySP2 is a Python package for processing data from Droplet Measurement Technologies’ Single Particle Soot Photometer (SP2).

The goals (and advantages) of PySP2 are:

* Bring DMT's processing code, originally written in IGOR, to open source in Python.

* Be interactable with Dask for parallelizing SP2 processing on clusters such as ARM Cumulus Cluster. 

PySP2 is capable of:

* Processing raw SP2 data from DMT in .sp2b format

* Generating wave properties from each channel

* Genetering particle mass and size distributions from the particle properties.

import act
import pysp2
import matplotlib.pyplot as plt
%pylab inline

%pylab is deprecated, use %matplotlib inline and import the required libraries.
Populating the interactive namespace from numpy and matplotlib

Loading the SP2 data#

PySP2 has two procedures for reading in the probe configuration in a .ini file as well as reading an .sp2b file. Here, we load a sample time period from the SP2.

config = pysp2.io.read_config('20181110/20181110114046.ini')
in_sp2b = pysp2.io.read_sp2('20181110/20181110x001.sp2b', arm_convention=False)

list(config.keys())

['DEFAULT',
 'Versions',
 'Program',
 'Controllers',
 'Alarms',
 'Alicat Flow Controller',
 'Control',
 'SPAT',
 'Acquisition',
 'Digital',
 'Laser',
 'Analog Input',
 'Housekeeping',
 'Streaming Data',
 'Flow Meters',
 'Calculated Channels',
 'Calculations',
 'Missing Value']

SP2 raw outputs#

The SP2 has 8 different photodiode detector arrays that measure the scattering and incandescence signals from the laser shining onto a partilcle that enters the beam. A refractory black carbon particle would, first, scatter the laser beam back to the detector. Applying Mie theory to the scattering signal gives us the approximate diameter of the particle entering the beam. Channels 0 (high gain) and 1 (low gain) measure the strength of this signal along a photodiode array that is aligned along the direction of the sample flow. In essence, the position along the photodiode array is proportional to the time at which the particle scatters light after it enters the laser beam. The low gain channel signal is useful for when the high gain channel is saturated from large particles entering the SP2 sample volume.

However, after the non-black carbon material coating the particle is evaporated by the heat from the laser, the black carbon proceeds to incandesce. The SP2 then records the signal into ch4 (high gain channel) and ch5 (low gain channel). This is done for one of every x number of partciles, where this x can be set by the user while operating the SP2. Most of the time, this is set to 5, since recording every particle would take up too much storage and processing time.

These raw waveforms for each particle are stored into an xarray dataset, shown below.

Let’s take the first particle in this file and plot the raw signals recorded by the SP2. First, we will look at the high gain scattering (ch0) and low gain scattering (ch4) signals.

in_sp2b["Data_ch0"].sel(event_index=10).plot(label='Ch0 voltage')
in_sp2b["Data_ch4"].sel(event_index=10).plot(label='Ch4 voltage')
plt.ylabel('Scattering Ch0 voltage [W]')
plt.xlabel('Diode #')
plt.legend()

<matplotlib.legend.Legend at 0x15fe23710>

../../../_images/73686375f98ddee8a386f136c40b65e29c1cebb1ae1adacf77b21abf81d07430.png

As we can see here, the peak of the scattering signal (ch0) is at around diode 32 with a Gaussian shape. In the low gain scattering channel (ch4), we see a similar peak with a much smaller magnitude since these diodes are less sensitive to the amount of scattered light from the laser beam.

Obtaining particle size distributions#

In this section, we’ll go over how to calculate and plot mass and number concentrations and size distributions. Often, the SP2 will need to be calibrated in order to ensure that mass and number size distributions are properly calculated. Thankfully, PySP2 has a DMTGlobals class that enables seamless loading of calibration files.

DGlbals = pysp2.util.DMTGlobals('20181110/Unit24CAL_Aldine_final.txt')

Let’s process the particle sizes and masses using the calibration we loaded.

psd = pysp2.util.calc_diams_masses(part_stats)

/Users/rjackson/PySP2/pysp2/util/particle_properties.py:38: RuntimeWarning: All-NaN slice encountered
  PkHt_ch0 = np.nanmax(np.stack([input_ds['PkHt_ch0'].values, input_ds['FtAmp_ch0'].values]), axis=0)
/Users/rjackson/PySP2/pysp2/util/particle_properties.py:39: RuntimeWarning: All-NaN slice encountered
  PkHt_ch4 = np.nanmax(np.stack([input_ds['PkHt_ch4'].values, input_ds['FtAmp_ch4'].values]), axis=0)

Number of scattering particles accepted = 638017
Number of scattering particles rejected for min. peak height = 26642
Number of scattering particles rejected for peak width = 12482
Number of scattering particles rejected for fat peak = 0
Number of scattering particles rejected for peak pos. = 30819

Let’s view the number concentrations. Since we have only processed about five minutes of data for this example, we will zoom into the processed data.

psds.NumConcIncan.plot()

[<matplotlib.lines.Line2D at 0x15b356e50>]

../../../_images/3f27f14cb2ad652e7541ab8303878bcce20ce7d1748c2499f33f31c6462f11d3.png

psds.NumConcScat.plot()

[<matplotlib.lines.Line2D at 0x165e82a50>]

../../../_images/13eaca38f9cfaf06b72ec623ff37767ea0a9fb77aee1389895d6e547528acc53.png

Let’s take a look at the particle size distributions!

psds.ScatMassEnsemble.T.sel(time=slice('2018-11-10T12:10:00', '2018-11-10T12:50:00')).plot(vmin=0, vmax=0.1, cmap='Spectral_r')

<matplotlib.collections.QuadMesh at 0x166b06190>

../../../_images/bc590785e6b77f31d4920225b3fa2f65772a15ec5da6e5a94ae414ea30d8d6e8.png

psds.IncanMassEnsemble.T.sel(time=slice('2018-11-10T12:10:00', '2018-11-10T12:50:00')).plot(vmin=0, vmax=0.01, cmap='Spectral_r')

<matplotlib.collections.QuadMesh at 0x1665c6610>

../../../_images/4729340aabedfe5fe18966072c86357d18d320ea1c624d12f2be3263d9b6cffd.png

PySP2

Contents

PySP2#

Loading the SP2 data#

SP2 raw outputs#

Obtaining waveform statistics#

Reading and plotting housekeeping data#

Obtaining particle size distributions#