Automatic Readiness and Training Systems Enhanced with Artificial Intelligence, Machine Learning, and Optimization

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Advanced Computing and Software;Integrated Sensing and Cyber;Trusted AI and Autonomy OBJECTIVE: Develop an automated, artificially intelligence-enhanced system for scalable effective training and monitoring of skilled physical tasks. Proposed solution should be demonstrated in a domain of interest to the Department of Defense such as maintenance, operation of military equipment, surgery, manufacturing, or medical tests. DESCRIPTION: Ensuring the readiness of the military requires the execution of a large number of skilled physical tasks in a variety of areas such as aircraft maintenance, operation of detection systems, medical care, production of advanced materials or equipment, and medical tests. Thus, it is critical to train and certify personnel to perform these tasks and to provide monitoring that supports ongoing quality improvement, through both individual feedback on performance and data collection for optimizing how the task is performed. The training environments range from centralized facilities to infrastructure-poor and geographically isolated locations with limited network connectivity. Historically, training and performance feedback has been provided via direct supervision of trainees by experts. This approach is limited by the number of qualified experts available and the geographical location of trainees. A single expert can train and supervise a limited number of personnel at a time and must travel to the trainees' location. This prevents rapidly scaling capabilities relying on skilled physical tasks. When the experts available are too few to provide adequate ongoing supervision, it can also result in low-quality work-product. Moreover, recording data on how the work is performed can be time consuming, requiring tradeoffs between data collection and the amount of work accomplished. Finally, when experts leave their roles, knowledge is lost regarding how to perform specific tasks. While some training can be provided via written documents and videos alone without direct supervision, the quality of this training is low and such methods do not provide effective supervision or data collection. The Navy seeks an approach for scalable effective training and monitoring of physical tasks that are mission critical and require extended concentration. This approach should be able to rapidly train and monitor thousands of personnel across many geographical areas with only one expert. It should use video and audio inputs to automatically record and process information about the work done to support individual feedback, optimization, and integration with other efforts that would benefit from information about work status. User input should not be required. It is envisioned that artificial intelligence and machine learning will play a key role in this approach as recent advances in these areas are poised to support automating many of the tasks that have historically been performed by individual experts. Work produced in Phase II may become classified. Note: The prospective contractor(s) must be U.S. owned and operated with no foreign influence as defined by 32 U.S.C. 2004.20 et seq., National Industrial Security Program Executive Agent and Operating Manual, unless acceptable mitigating procedures can and have been implemented and approved by the Defense Counterintelligence and Security Agency (DCSA) formerly Defense Security Service (DSS). The selected contractor must be able to acquire and maintain a secret level facility and Personnel Security Clearances. This will allow contractor personnel to perform on advanced phases of this project as set forth by DCSA and ONR in order to gain access to classified information pertaining to the national defense of the United States and its allies; this will be an inherent requirement. The selected company will be required to safeguard classified material during the advanced phases of this contract IAW the National Industrial Security Program Operating Manual (NISPOM), which can be found at Title 32, Part 2004.20 of the Code of Federal Regulations. PHASE I: Provide a viable concept with detailed methodology to address the general training problem in the Description. A specific area of training in a domain of interest to the Department of Defense should also be identified, perform testing, and provide preliminary results. The concept should be generalizable beyond one specific area of training. PHASE II: Develop a prototype for testing, demonstrating, and validating that the method improves performance for novice individuals on a specific area of training. Depending on the specific area chosen, this could result in the some of the work being classified. It is probable that the work under this effort will be classified under Phase II (see the Description section for details). PHASE III DUAL USE APPLICATIONS: Transition the prototype to a mature capability that will be used for a broad range of different areas of training. The ideal end-state should be usable on handheld devices to leverage the video and audio technology there to provide input to the automated system, which would allow for feedback. Industrial and government sectors are expected to procure varying levels of this technology depending on the level of sophistication required by their training requirements. REFERENCES: 1. Banerjee, A. et al. AI Enabled Tutor for Accessible Training. Artificial Intelligence in Education, Vol. 12163, 2020, pp. 29-42. doi 10.1007/978-3-030-52237-7_3 2. Yazdi, M. Acquiring and Sharing Tacit Knowledge in Failure Diagnosis Analysis. Journal of Failure Analysis and Prevention, Vol 19(2), 2019, pp, 369-386. doi.org/10.1007/s11668-024-01904-y 3. Yilmaz, R; Bakhaidar, M and Del Maestro, RF. Real-Time multifaceted artificial intelligence vs In-Person instruction in teaching surgical technical skills: a randomized controlled trial. Scientific Reports, 14(1), 2024. doi.org/10.1038/s41598-024-65716-8 4. Brooks Babin, LisaRe and Garven, Alice J. (Sena). Tacit Knowledge Cultivation as an Essential Component of Developing Experts: A Literature Review. Journal of Military Learning, April 2019. https://www.armyupress.army.mil/Journals/Journal-of-Military-Learning/Journal-of-Military-Learning-Archives/JML-Apr-2019/Babin-Garvin-Tacit-Knowledge/ 5. National Industrial Security Program Executive Agent and Operating Manual (NISP), 32 U.S.C. 2004.20 et seq. (1993). https://www.ecfr.gov/current/title-32/subtitle-B/chapter-XX/part-2004 KEYWORDS: Artificial intelligence; machine learning; applied optimization; training