PROJECTED CMMC LEVEL REQUIREMENT
Level 2 (Self)
TECHNOLOGY AREAS
Air Platform
|
Human Systems
|
Information Systems
MODERNIZATION PRIORITIES
Trusted AI and Autonomy
KEYWORDS
Mission Debriefing; After-Action Review; Autonomy; UX/UI; Human-Machine Teaming; Trust
OBJECTIVE
Future Autonomous Collaborative Platforms (ACPs) will introduce AI-enabled uncrewed aircraft into the fleet. These platforms will assume significant tactical decision-making responsibilities and operate alongside traditional crewed aircraft. This paradigm shift complicates knowledge elicitation for post-mission debriefing, as it necessitates understanding both human and autonomous aircraft decision-making processes. This introduces a new research challenge: effectively logging the necessary information from human and ACP decision-actions for debriefing and presenting it to warfighters through innovative human-machine interfaces (HMIs). The primary objective of this topic is to prototype and develop debriefing approaches that effectively fuse the decision-making chains of both human operators and multiple autonomous ACPs, presenting that information clearly and concisely.
ITAR
The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with section 3.5 of the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.
DESCRIPTION
Mission debriefing for manned and remotely piloted aircraft (RPA) and crewed aircraft in military operations is currently conducted manually by warfighters. This typically involves verbal communication and classroom-style discussions, with little to no AI or software assistance for reflecting on missions, identifying lessons learned, or pinpointing areas for improvement. As pilots are the primary tactical decision-makers, verbal communication sessions are essential for eliciting and understanding their decision-making processes. As teams of ACPs begin making tactical decisions with a high level of autonomy, it is unknown what information needs to be logged during mission and how that information should be displayed so that the warfighter can audit and understand after mission debriefing, what decisions, tactics, techniques, and procedures (TTPs) the autonomous systems acted on. This topic looks to advance existing debriefing tools for replaying mission execution and enhance them with additional functionality targeting debriefing of autonomous ACPs.
A secondary focus of this topic is to identify data input requirements from autonomy that would be necessary for support debriefing of autonomy. Modern methods for autonomous decision-making tend to employ black-box deep learning algorithms with limited transparency, leading to lack of trust and assurance that autonomous agent decisions comply with the Law of Armed Conflict (LOAC). XAI (Explainable Artificial Intelligence) is actively researching techniques to make black box models more understandable while other areas are using more transparent symbolic methods that are rooted in explicit rules to perform reasoning and problem-solving. An ACP will likely include a combination of inherently explainable and low transparency algorithms for different decision-making processes. Information needs for debriefing that will be identified in this topic should guide autonomy development with regards to autonomy logging/reporting for debriefing and algorithm practicality.
A consideration for DP2 participation is the demonstration of an existing debriefing tool that the proposer has developed that is used in military operations or that it is being developed under a recognized US DoD program. This will allow for a solid foundation for which CHORD can build upon that focuses specifically on debriefing of human machine teaming for ACPs. An expectation of common debriefing functionality such as data playback, a digital map display, timeline, event logs, and data visualization of vehicle fuel, health, and status will be necessary for DP2 consideration. It is not essential that the existing debriefing tool has been applied to unmanned systems, and debriefing tools in non-air or crewed vehicle domains will be considered.
While logging and video playback of ACP mission execution are critical components of debriefing functionality, they will likely be inadequate for truly understanding ACP decision-making. Software analytics, AI tools, and novel HMI designs will be necessary to answer key questions, such as: What tactical decisions did the ACP make? When were these decisions made? What was the rationale or considerations behind the ACP's decisions? As ACPs assume greater responsibility in tactical decision-making, it is crucial to conduct research and develop software tools that enable warfighters to understand and trust these autonomous systems.
Core Research Questions:
What types of information must be logged and exchanged between ACPs and the warfighter during post-mission debriefing to support transparency and trust in autonomous operations?
Do current government reference architectures and standards adequately support the information exchange requirements for debriefing ACP teams?
How should information from ACPs be structured and visualized within the HMI to align with warfighter cognitive models and situational awareness needs?
What HMI features for debriefing best support comprehension of ACP autonomy decision chains, contextual reasoning, and deviations from expected behavior?
What types of software or AI-enabled analytics tools would be most useful to summarize, explain, and visualize autonomous decision-making by ACPs?
PHASE I
This topic is for Direct to Phase II (DP2) submission. The Department of Air Force will accept Direct to Phase II proposals for the cost of up to $1,700,000 for a 18-month period of performance.
To submit a DP2 proposal, proposers must justify the scientific and technical merit, feasibility, and potential military and/or commercial applications of their proposed work. This justification requires detailed documentation, such as technical reports, test data, prototype designs/models, and achieved performance goals/results. The following are prerequisites for DP2 consideration:
Proposer must demonstrate an existing debriefing tool or after-action review (AAR) tool that they have developed, either already deployed in military operations or developed under a recognized U.S. DoD program.
Though most of the work will be executed at UNCLASSIFIED or CUI, there is the potential for work to be executed at a SECRET level. Proposers must be able to acquire and maintain a secret level facility and Personnel Security Clearances.
Proposer must demonstrate knowledge of and access to subject matter expertise regarding Air Force doctrine, tactics, techniques, and procedures.
Proposer must exhibit strong technical merit and possess a team with expertise in software development, architecture, deployment, front-end web development, autonomous systems, government reference architectures, and artificial intelligence.
Proposer must demonstrate a clear understanding of the complexities involved in debriefing missions where both humans and machines are making decisions.
Proposer must present a strong commercialization strategy, including a well-defined plan for bringing a debriefing software solution to market.
PHASE II
Provide a design, prototype, and integrated demonstration of a debriefing tool built on a government reference architecture. Potential features to explore for debriefing UI includes: components that give insight into ACP team decision making, visualize ACP team and individual state and performance, provide situation awareness into the environment the ACPs are interacting with, allows conversing with the debriefing tool using conversation AI (e.g., LLMs), visualize uncertainty and mission anomalies, and allows users to explore what-if scenarios by modifying key parameters of the mission (e.g., number of ACPs, ACP payloads, weather conditions) and re-run the mission in a simulated environment to see how the ACP would have responded are all of interest in this phase 2 topic. Performers will have opportunities to develop other transformational debriefing concepts for debriefing teams of ACPs.
Deliverables include:
A government reference architecture compliant debriefing tool developed in a modern web-based UI development framework.
An integrated demonstration of the debriefing tool with multi-vehicle simulation.
Technical Design Document (TDD) of debriefing software including:
Technical whitepaper detailing key findings.
UML software documentation
UX/UI wireframes and mockups (e.g., Figma, Balsamiq, Adobe XD).
4. Government reference architecture change proposals addressing identified gaps regarding the interfaces between autonomy and mission debriefing.
PHASE III DUAL USE APPLICATIONS
Developing mission and autonomy debriefing software through this SBIR will directly benefit future Autonomy-Capable Platforms (ACPs) reliant on autonomous decision-making. The resulting robust and versatile solution will enable CHORD to dramatically improve the efficiency and trustworthiness of post-mission analysis for both military and commercial autonomous systems.
REFERENCES
Bhuyan, B. P., Ramdane-Cherif, A., Tomar, R., & Singh, T. P. (2024). Neuro-symbolic artificial intelligence: a survey. Neural Computing and Applications , 36 (21), 12809-12844.
Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., ... & Ranjan, R. (2023). Explainable AI (XAI): Core ideas, techniques, and solutions. ACM computing surveys , 55 (9), 1-33.
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., ... & Hussain, A. (2024). Interpreting black-box models: a review on explainable artificial intelligence. Cognitive Computation , 16 (1), 45-74.; Johansen, B. I., & Fredborg, B. (2000). Mission debriefing system
Loyola-Gonzalez, O. (2019). Black-box vs. white-box: Understanding their advantages and weaknesses from a practical point of view. IEEE access , 7 , 154096-154113.
Mastaglio, T. W., Jeffery Wilkinson, J., & Jones, P. N. (2011). Current practice and theoretical foundations of the after action review (No. TRAR0075111508). US Army Research Institute for the Behavioral and Social Sciences.
Oxford Analytica. (2023). Crewed and uncrewed combat aircraft will collaborate. Emerald Expert Briefings , (oxan-db).
QUESTIONS & ANSWERS
NOTE:
To ask a question, you must
log in or create an account
for the DSIP.
