FA875024CB130

Small Business Innovation Research Program (SBIR) Phase II

Explainable Large Language Models Generating Responses Organized and Verifiable with Evidence (ELLMGROVE)

Trustworthy Generative Artificial Intelligence (GenAI) to Structure Data and Deliver Accurate Insights of Command, Control, Communication and Computer (C4) Systems

Assured Information Security, Inc., in collaboration with Georgia Tech Research Institute and Infinity Labs, LLC, proposes Explainable Large Language Models Generating Responses Organized and Verifiable with Evidence (ELLMGROVE), a research effort to develop explainable Large Language Models (LLMs) as Reinforcement Learning (RL) agents to support Command, Control, Communication, and Computer (C4) systems. Since the release of GPT-2 [1], instruction-tuned [2] LLMs have revolutionized the field of Natural Language Processing (NLP), achieving state-of-the-art (SOTA) performance on a variety of tasks. These capabilities make LLMs a promising technology for augmenting C4 systems by providing a means to analyze new intelligence as it arrives and providing actionable recommendations based on emerging situational awareness. In situations where operators, warfighters, and/or commanders use C4 systems, they may only have a short window in which to act on new intelligence, as it becomes stale, useless, or otherwise not actionable in a matter of hours. In such settings, any Artificial Intelligence and Machine Learning (AI/ML) system intended to expedite the exploitation of incoming intelligence must have strong zero-shot capabilities, as the short window in which to act precludes online learning or finetuning. Despite the remarkable abilities of LLMs, deploying them in DoD-relevant settings requires overcoming critical hurdles. Like other AI/ML models, LLMs are black boxes, and there is not a clear understanding of how they can perform linguistic tasks or understand natural language. Moreover, just as clever prompting can elicit reasoning in LLMs [5], it can also manipulate a model into violating the guardrails intended to prevent it from producing offensive, sensitive, or otherwise inappropriate responses [12]. This vulnerability makes it difficult to use LLMs in settings involving classified or otherwise sensitive data as the model may, for example, divulge sensitive information to an operator. LLMs have demonstrated remarkable problem-solving skills through their application to RL. For example, LLMs can reason over long-term objectives, a perennial challenge in RL, while a traditional RL agent focuses on short-term skill building through its interactions with the environment [14]. Alternatively, an LLM can itself become an RL agent if prompted with a description of the environment, the objectives it needs to solve, and guidance on how to reason about its observations [15]. In the context of C4 systems, an LLM-based RL agent can receive observations in the form of incoming intelligence and generate recommendations about how best to respond to them. Importantly, if the agent can cite accurately to the specific information that justifies its responses, including incoming intelligence, military doctrine, policies, etc., the operator can have greater confidence and trust in the LLM and its responses.

The goal of phase II is to continue the R&D efforts initiated in Phase I. Funding is based on the results achieved in Phase I and the scientific and technical merit and commercial potential of the project proposed in Phase II.

AF241-D013

F2D-11676

24.1

Simon Khan