N6833524C0096

Small Business Innovation Research Program (SBIR) Phase II

TALON: Multi-INT Metadata Extraction for Threat Detection

Artificial Intelligence-Driven Multi-Intelligence Multi-Attribute Metadata Enabling All-Domain Preemptive Measures

Identifying and tracking developing threats across a large and growing set of data sources remains a critical unsolved problem. Currently, multi-modal data (images, video, audio clips, satellite images, etc.) as well as OSINT data (social media platforms like Twitter, Telegram, etc.) must be manually inspected by an analyst to extract relevant metadata, such as who , what , where , when , and why . Subsequent correlation across multiple modalities, required to identify and track developing threats in operational domains, is also a manual process. Recent advances in computer vision, machine learning (ML), and natural language processing (NLP) technology have resulted in the creation of innovative algorithms capable of assisting an analyst in understanding, cataloging, and characterizing multi-modal content automatically. However, these advances have seen limited application to defense and intelligence problems. Vision Systems Inc. (VSI) with support from STR proposes TALON, an innovative multi-INT analysis framework designed to generate information-rich metadata across multi-INT data streams and use the context provided by this metadata to detect emerging threats. In Phase I, TALON leveraged advances in ML and NLP to discover events from publicly available social media and online sources and produce flexible and semantically rich natural-language descriptions of associated multi-modal data. The proposed SBIR Phase II program will build on Phase I results, enhancing and expanding metadata generation capabilities, analyzing metadata using the latest in state-of-the-art NLP systems such as Large Language Models (LLMs) models, and considering operational data sources of interest to potential stakeholders across the defense and intelligence community. The VSI and STR team is uniquely suited to this project, offering unparalleled expertise in machine learning, natural language processing, and remote sensing; extensive experience in the development of innovative software solutions for challenging defense and intelligence problems; and a thorough understanding of the needs of image analysts across the defense and intelligence community.

The goal of phase II is to continue the R&D efforts initiated in Phase I. Funding is based on the results achieved in Phase I and the scientific and technical merit and commercial potential of the project proposed in Phase II.

N222-118

N222-118-0049

22.2

Allen N Moshfegh