Formatted Title
From Spreadsheets to Scripts: Challenges, Strategies, and Lessons Learned in the Development of an Environmental Data Analytics Practice
Background/Objectives
The environmental data analytics landscape is changing. Emerging professionals have increased capabilities in programming and data automation. Increased availability of interactive dashboard tools has generated interest from clients and stakeholders who want to modernize their decision-making workflows. Tools that automate environmental remediation site data access, analysis, and visualization are more widely accessible. Complex legacy sites such as military bases now have decades of data whose utility for decision-making could be improved if integrated using modern and advanced analytics. However, there are numerous challenges in this rapidly growing area. There is an overwhelming array of terminology and tools associated with environmental data analytics. Workflows dominated by spreadsheets need to be overhauled and redesigned into different formats, such as R and Python scripts. Quality control practices need to be redefined. As we adapt to client expectations and improve the economy, efficiency, and efficacy of our daily work, we must collaborate across the industry to develop the environmental data analytics practice, while still accommodating industry professionals less conversant in software development and practices.
Approach/Activities
Our approach was to establish a sub-discipline to tackle challenges associated with emerging environmental data analytics capabilities. We gathered professionals across the disciplines of business technology, engineering, environmental science, data management, computer science, and research and development. Our initial activities included discussions to 1. define and demystify terms, 2. describe the interplay between engineering, environmental, and computer sciences, and 3. understand available tools for modernizing various data analytics applications. Out of this foundation emerged additional initiatives, including development of improved data management and analytics practices; education and resources for staff, clients, and project managers; updates to quality procedures and workflows; and adjusting contracting language and proposal content to be inclusive of digital tool development and maintenance. We continue to evolve in this space.
Results/Lessons Learned
Modernizing environmental data analytics is integral to projects with high volumes of complex data requiring frequent reporting, reproducibility, and client and stakeholder engagement. Firstly, for effective communication in the evolving digital space, it is first important to establish common terminology, and we will share some of our definitions. Secondly, for effective collaboration, it is important to understand the connections across the disciplines, and we will share our interdisciplinary network. Thirdly, to best serve our clients and engage our teams, it is important to stay apprised of available tools and initiate development of additional customized and flexible tools. Tools may include spreadsheets, programming languages, proprietary (e.g., Microsoft Power BI, Automate, the CAST Platform) and open-source (e.g., Plotly-Dash and R Shiny) dashboard tools, relational databases, numerical models, and geographic information system (GIS). It is important to integrate the appropriate tools based on project objectives, and we will share examples from dashboard development and code standardization for recent projects. Key components of an environmental data analytics practice include staff and client education, centralized communication platform for analytics staff, and research and development. While these continue to evolve, our focus growth areas also include intellectual property reuse and protection, quality management, and management of cloud services. We will share some of our successes and challenges in transitioning our environmental practice into a digital space. Ultimately, it was critical to establish an organizational body to advance our environmental data analytics practice.