FA864922P0680

Small Business Innovation Research Program (SBIR) Phase II

Offline Learning and Counter Artifical Intelligence for Autonmous Aircraft Combat Operations

Reinforcement learning (RL) consistently produces controllers that exceed human performance on complicated tasks in control and strategy. Despite this promise, their widespread use is limited by a few important problems. First, they are incredibly compute intensive. Popular RL algorithms do not allow the reuse of data during their learning. This problem results in inflexible controllers because controllers cannot be easily modified for a new task, and this constraint limits the number of controllers that can be deployed because each controller takes immense resources to create. Last, this inflexibility results in suboptimal controllers because controllers cannot easily learn from different sources of data, which includes examples from expert human operators. The second issue is that most controllers learned by RL are susceptible to counter AI attacks, which can force a controller to fail catastrophically. While this issue is not important for applications where environments are reliable and fair, DOD applications cannot operate under these assumptions. Toyon Research Corporation will develop new training methodologies that enable data reuse and the ability of RL algorithms to learn from related tasks. We will also research Counter AI attacks, and will develop methods to make controllers learned with RL more robust to these attacks.

The goal of phase II is to continue the R&D efforts initiated in Phase I. Funding is based on the results achieved in Phase I and the scientific and technical merit and commercial potential of the project proposed in Phase II.

AF211-CSO1

FX211-CSO1-0910

X21.1

Marcella Lindbery