Formatted Title
Considerations when Building Data Pipelines for Machine Learning Applications: Practical Examples and Tools
Background/Objectives
Understanding the practical considerations for implementing machine learning (ML) requires understanding the history and purpose of ML. The term “Machine Learning” was coined in the 1950s by scientists at IBM which described computer systems capable “discovering” their “own” algorithms. While originally considered synonymous with artificial intelligence (AI), the two fields of study bifurcated in the 1980s. AI research focused on reasoning through problems with “if-then” style reasoning (e.g., expert systems and inductive logic) while ML focused on using probabilistic inference to discover patterns between data. Today, ML is considered a sub-field of AI with a more limited scope, but with a greater ability to interpret results. Over the last decade, ML modelling tools became more readily accessible as computational power increased and open-source programming flourished. The “model agnostic” nature of ML makes it an immensely valuable tool for discovering the relationship between target data and parameters of interest, but requires thoughtful implementation to have value. Today, there is a consensus that reliable data, domain area expertise, and programming knowledge are all equally important aspects of using ML tools. This presentation explores the proper ways to generate reliable ML- based insights by focusing on domain area expertise and modern data management techniques at the inception of a project.
Approach/Activities
The “garbage in – garbage out” paradigm is introduced to explain how improperly managed data can result in trained ML models which produce undesirable outcomes. These outcomes can include artificially inflated/deflated accuracy scores from inappropriate data splitting, bias in results from “hidden” variables, and mistaken objectives producing inaccurate results. The concept of data pipelines is introduced as a method for properly acquiring, managing, cleaning, and interpreting data. Example data pipelines are demonstrated using synthetic environmental data from multiple scenarios (i.e., PFAS impacted groundwater site, traditional chlorinated solvent location). The data pipelines generally consist of five stages: 1) extracting raw data from complex site locations; 2) validating and transforming the raw data; 3) organization of data into a relational “data warehouse” of varied data (i.e., chemistry, geospatial, geotechnical, and climatology data); 4) importing organized data into analysis software for interpretation; and 5) recurring analysis quality control steps for error detection. While the analysis (step 4) of the data pipeline can take many shapes, this case study will focus on practical examples of ML based analysis for environmental applications.
Results/Lessons Learned
A brief history of ML is presented to introduce the purpose of ML models and why the “garbage in – garbage out” paradigm is important. By constructing well thought out data pipelines upfront, environmental engineering/management practitioners can reduce the possibility of “garbage in” errors. The increased upfront labor for constructing a robust data pipeline is offset by the increased time savings during the analysis stage of work. We emphasize that proper construction of data pipelines requires input from practitioners with expertise in the domain area of study. Without expert input, data pipelines can miss important variables, overemphasize other variables, or have ML algorithms which are improperly suited to study the specified problem. Examples of frequent ML modelling errors are shown to give context to the larger problem. Examples of ML applications relevant to the field of study show how insights from ML are unique compared to traditional mechanistic modelling or statistical approaches. A suite of open‑source ML tools are highlighted for the benefit of practitioners. Overall, unlocking the power of ML based approaches will require thoughtful data management practices.