Universal Naval AI CORe eNvironement (UNACORN) Data Engineering Pipeline

The scope of this acquisition tasks the contractor in supporting the development and on-going operation of UNACORN Data Engineering pipelines that use open source data engineering software which will be composed into automated workflows to enable the ability to develop and integrate new and emerging technologies. These technologies will be used to extract, process, analyze, transform, secure, and preserve Naval information in the form of structured, unstructured, and streaming data sets. These automated workflows will be required to support both historical data sets and real-time streaming data sources for processing afloat data and to produce mission products (e.g. vessel track reports) from afloat declarations. The contractor shall use best practices and methods from commercial industry to perform the extraction, processing, analysis, transformation, and feature enhancement of the data sets for use by Artificial Intelligence (AI) and Machine Learning (ML) models and algorithms developed by other entities within the UNACORN ecosystem. The contractor shall support the development and operation of a real-time data pipeline monitoring and operations dashboards in order to determine Data Quality and Data Drift and take corrective actions on the data sets. The UNACORN Data Engineering Pipeline services will be developed in order to leverage fluid environments of cloud compute and storage, hardware, and cloud edge devices and be transferrable between each of these environments.