Formatted Title:

Developing a Robust Sampling Program Utilizing Geostatistical Analysis, Conditional Simulations and Receiver Operating Characteristic (ROC) Curves

Background/Objectives:

Operable Unit (OU) 4 of the Hudson River PCBs Superfund Site consists of over 40 miles of floodplain soils adjacent to the Upper Hudson River. A comprehensive study of OU4 is under way to better understand the distribution of PCBs in floodplain soils. To support selection of the most appropriate remedy at large sites like OU4, it is advantageous to have a sampling program that is both cost-effective and efficient at correctly identifying areas of elevated concentrations that require remediation. This paper introduces a workflow for optimizing a soil sampling program to maximize correct classification of PCB concentrations relative to a threshold. The workflow combines geostatistics, conditional simulation, and receiver operating characteristic (ROC) curves. The results can be applied at contaminated soil and sediment sites to aid in optimizing a sampling program for a particular cleanup level and improve investigation cost-effectiveness.

Approach/Activities:

The key input into the workflow is the distribution and spatial correlation structure (i.e., semi-variogram) of the contaminants of concern (COCs) of interest. Sample data from OU4 floodplain soils was used to develop the geostatistical model for PCBs. This model was used to develop many (i.e., hundreds) of conditional simulations (maps), which are unique realizations of the distribution of PCB concentrations that remain faithful to the underlying spatial structure of the data. For each conditional simulation, various sampling permutations were evaluated by varying: 1) sampling density, 2) decision unit size and 3) hypothetical PCB cleanup thresholds. For each permutation, the PCB concentrations of the sampled decision units were used to estimate the PCB concentrations in the unsampled decision units. The estimated concentrations were compared to the true (simulated) concentrations to develop a confusion matrix/ROC curve summarizing performance. The ROC curves visualize how well a sampling design can correctly identify areas where PCB concentrations exceed (and do not exceed) a threshold (true positives and true negatives, respectively) while minimizing incorrect identification.

Results/Lessons Learned:

Results demonstrate that the lower the cleanup threshold relative to the site-wide mean PCB concentration, the higher the sensitivity achieved (higher probability of correctly identifying areas above threshold), but at the cost of a lower specificity (low probability of correctly identifying areas of below threshold). The optimal concentration threshold to balance sensitivity and specificity may not align with pre-determined cleanup levels, and a trade-off may be required between increasing the probability of correctly remediating areas above the cleanup threshold, but also increasing the probability of remediating areas that are not above the threshold. This workflow can aid in the following: (1) development of an appropriate sampling program (2) support correctly identifying areas that require remediation, and (3) provide insight into a remedial design’s cost-effectiveness (i.e., amount of effort spent remediation areas that do not require remediation).

Jill Gulczewski

Affiliation

Refine Results

Developing a Robust Sampling Program Utilizing Geostatistical Analysis, Conditional Simulations and Receiver Operating Characteristic (ROC) Curves