Numerics-Informed Neural Networks (NINNs)

RT&L FOCUS AREA(S): Artificial Intelligence/ Machine Learning TECHNOLOGY AREA(S): Chem Bio Defense; Information Systems OBJECTIVE: DTRA has a need to perform high-fidelity CFD modeling of agent defeat phenomenology and associated test and evaluation activities in order to quantify and increase the accuracy of hazard source predictions for counter weapons of mass destruction (C-WMD) defeat and deny tactics. These simulations are technically and computationally challenging due to the long-time duration of interest (weapon detonation through stabilization of plume), the stochastic nature of fragmentation and turbulent mixing phenomena, the temperature dependency of thermal neutralization mechanisms, and the relatively stiff chemical kinetics models. The objective of this topic is to improve the computational efficiency of the chemical kinetics models for chemical weapon agents and simulants by investigating and developing Numerics-Informed Neural Networks (NINNs). This topic explores the premise that simply using the residual of the PDE as in Physics-Informed Neural Networks (PINNs) is not optimal. One might instead use directly the numerical schemes which are employed to integrate the PDEs in time. This leads naturally to numerics-informed neural nets (NINNs). DESCRIPTION: The last decade has seen a tremendous amount of activity and developments in the field of deep neural networks (DNNs). When trying to apply these to physics governed by partial differential equations (PDEs), traditional DNNs have been supplemented' or informed' with the underlying physics, leading to physics-informed neural nets (PINNs). This topic explores the premise that simply using the residual of the PDE (as in PINNs) is not optimal. One might instead directly use the numerical schemes which are employed to integrate the PDEs in time. This leads naturally to Numerics-Informed Neural Networks (NINNs). To leverage the ongoing research momentum in Artificial Intelligence and Machine Learning, DTRA seeks innovative ideas for replacing the PDE residuals used for PINNs by the discrete time stepping increments of numerical integrators. Phase I development must demonstrate a NINN approach for local residuals (e.g., chemically reacting flows) and non-local residuals (e.g., PDEs with spatial derivatives). The new techniques should then be compared to PINNs and traditional DNNs. Phase II development will further optimize the NINN approach to extend the range of applicability to other problems. PHASE I: Define and develop NINNs for chemical reactions (CHEM-NINNs). Define and develop NINNs for PDEs with spatial derivatives. Investigate and validate NINNs and CHEM-NINNs by comparison of results with traditional DNNs and PINNs. PHASE II: Further develop, test and optimize the NINN approach to extend the range of applicability. Demonstrate use of NINNs on High Performance Computing (HPC) systems. Perform detailed comparisons with high-fidelity Computational Fluid Dynamics (CFD), Computational Chemistry application codes and observational data, to quantify speed and accuracy of the NINNs and CHEM-NINNs. Generalize and document for pre-commercial release. PHASE III DUAL USE APPLICATIONS: In addition to implementing further improvements that would enhance use of the developed product by the sponsoring office, identify and exploit features that would be attractive for commercial or other private sector HPC applications. The software developed for use in DTRA's very demanding application codes will be well suited, once refined, for use on more general HPC workloads. Investigate commercialization avenues that could include other government agencies, national labs, research institutes, and defense contractors. Develop a plan to enable successful technology transition at the end of this phase. REFERENCES: [1] Shin, Yeonjong , On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs},Communications in Computational Physics https://arxiv.org/pdf/2004.01806 ; [2] M Raissi, P Perdikaris, GE Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Journal of Computational Physics, 2019 https://www.osti.gov/servlets/purl/1595805 ; [3] Harbir Antil, Ratna Khatri, Rainald L hner, Deepanshu Verma, Fractional Deep Neural Network via Constrained Optimization https://arxiv.org/pdf/2004.00719 ; [4] Lars Ruthotto, E. Haber, Journal of Mathematical Imaging and Vision 2019 Deep Neural Networks motivated by Partial Differential Equations https://arxiv.org/pdf/1804.04272