HR001123C0118

Small Business Innovation Research Program (SBIR) Phase II

Workable Hierarchical Impersonation using Reinforcement Learning (WHIRL)

Workable Hierarchical Impersonation using Reinforcement Learning (WHIRL) will generate realistic synthetic data without artifacts at scale by utilizing hierarchical reinforcement learning and a hypervisor to allow for off-box execution of long-term goals, mid-term tasks, passed through a shim to a hypervisor that will execute them on the intended host. Team Cynnovative will use hierarchical reinforcement learning to simulate user behavior at the level a real user would: keyboard and mouse activity and observing a monitor. By simulating on real hardware and executing off-box, WHIRL enables the collection of the generated synthetic data via any traditional means a WHIRL user desires without introducing any artifacts or biases. User persona research for SUP is a method of understanding the characteristics and behavior patterns of specific groups of users to gain insights into the motivations, goals, and needs of these user groups and inform the design and development of effective cybersecurity strategies. This research seeks to understand how users interact with network systems, applications, and data to design policies that enable a user to operate successfully while maintaining a robust security posture. The autonomous agent for WHIRL is rewarded for taking actions to achieve a goal, such as browsing the web or using an excel sheet or operating in a terminal. Feedback from the environment informs the agent how well it accomplishes the task. A fully trained agent can act as a defined synthetic user enables the generation and collection of robust datasets representative of realistic user behavior. Team Cynnovative's solution will be operating on the raw pixels of a screen capture which puts reinforcement learning in a real-world domain with an observation space where it has succeeded in the past, effectively eliminating the simulation to real-world problems. Reinforcement learning can operate on pixel data and elicit realistic, emergent behaviors with ground truth. WHIRL operates off-box on real hardware (via hypervisor), enabling the collection of synthetic data the way the data is normally collected. This means that the users of the WHIRL system will not have to worry about learning how to collect via another platform but will enable them to leverage existing knowledge and tools and not worry about filtering for any artifacts or biases.

The goal of phase II is to continue the R&D efforts initiated in Phase I. Funding is based on the results achieved in Phase I and the scientific and technical merit and commercial potential of the project proposed in Phase II.

HR0011SB20234-02

D2D-0469

23.4

Matt Puglisi