Comprehensive Framework for Test & Evaluation of Digital Twins

TECHNOLOGY AREAS: Sensors; Electronics; Information Systems; Human Systems OBJECTIVE: The advent of digital twins and AI/ML presents a significant opportunity for the US Air Force to substantially improve Flight Test efficiency, viz. reduce both cost and time required for Test and Evaluation (T&E) of AF assets/capabilities. While significant effort and investments are made to create digital twins (DTs), there is no comprehensive framework to validate the completeness and accuracy of DTs. The purpose of this proposal is to address the need for such a capability. DESCRIPTION: Digital twins have emerged as powerful tools for simulating and analyzing the behavior of physical systems in aerospace and manufacturing applications (Madni et al., 2019). DTs represent a potential driver of efficiency in the physical T&E process- but only if the DT faithfully represents the functionality of the physical asset to be tested. Thus, it is incumbent upon the test community to ensure DT accuracy (Schluse et al., 2018). Along with details of data collection and analyses, validation of a DT's fidelity to the test item must be a part of any test plan. OSD Critical test areas: Successful development of a comprehensive, scalable framework for the verification and validation of a DT include: 1. Model-Based System Engineering (MBSE)- support improved model development. 2. Integrated Networks of Systems- demonstratable correct operation of integrated network systems 3. Artificial Intelligence/Machine Learning- application of these tools to advance OSD capability in system as well as DT development and test. Plainly, the framework proposed in this SIBR is essential if the US is to remain ahead of adversary use of AI in weapon development and innovation (Kana, 2020). We invite proposals from eligible vendors to 1. Create a comprehensive and scalable framework that test and evaluate the real-world applicability of digital twins, and 2. Demonstrate that such a framework can be applied to various Air Force digital twin applications. The framework should evaluate whether the digital twin can capture and characterize physical system behavior, interactions, and dependencies, necessary for defensible T&E results (e.g. Lin et al., 2021). APPROACH: Up to the contractor, MBSE is suggested, but not required. PHASE I: In Phase I, companies will develop a proof of concept and feasibility study that provide the conceptual design of a comprehensive test framework for digital twins. The conceptual design should include supporting literature for technical feasibility, showcase the technology's application opportunities to a broad base of DTs. At the end of Phase I, the company will be required to provide a concept demonstration of their technology to demonstrate a high probability that continued design and development will result in a Phase II mature product. PHASE II: In Phase II, a fully functional T&E platform should be developed that incorporates a framework that can be customized for analyzing digital twins. This analysis should demonstrate the completeness and accuracy of the digital twin. This means a scalable and comprehensive framework that results in defensible evidence of the accuracy of the digital twin output compared to flight test data. PHASE III DUAL USE APPLICATIONS: Complete the development of the technology developed in Phase II and produce prototypes to support further development and commercialization. Deliverables for Phase III include a market analysis report, a detailed business plan, a deployment and distribution plan, and documentation outlining potential adaptation and expansion opportunities. REFERENCES: 1. Madni, A. M., Madni, C. C., & Lucero, S. D. (2019). Leveraging Digital Twin Technology in Model-Based Systems Engineering. Systems, 7(1), Article 1. https://doi.org/10.3390/systems7010007 2. Schluse, M., Priggemeyer, M., Atorf, L., & Rossmann, J. (2018). Experientable Digital Twins Streamlining Simulation-Based Systems Engineering for Industry 4.0. IEEE Transactions on Industrial Informatics, 14(4), 1722 1731. https://doi.org/10.1109/TII.2018.2804917 3. Lin, L., et al. Uncertainty Quantification and Software Risk Analysis for Digital Twins in the Nearly Autonomous Management and Control Systems: https://www.sciencedirect.com/science/article/pii/S0306454921002383 4. Kana, E., (2020) AI Weapons in China's Military Innovation, Brookings Institute, https://www.brookings.edu/articles/ai-weapons-in-chinas-military-innovation/ KEYWORDS: Artificial Intelligence/Machine Learning, Network Command, Control and Communications