PROJECTED CMMC LEVEL REQUIREMENT
Level 2 (Self)
TECHNOLOGY AREAS
Sensors
MODERNIZATION PRIORITIES
Trusted AI and Autonomy
KEYWORDS
Scene generation; artificial intelligence; radio frequency; electro optical; infrared; multimodal; synthetic data; geo-specific
OBJECTIVE
The objective is to develop a capability for generating geo-specific, sensor-independent scenes for multimodal (RF and EO/IR) synthetic data generation by leveraging geo-spatial information, time-of-day, seasonal data, and measured databases, overcoming limitations in existing models and radiometric data.
ITAR
The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with section 3.5 of the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.
DESCRIPTION
The DoD requires large-scale, high-fidelity background scenes to advance autonomous systems and Artificial Intelligence and Machine Learning (AI/ML) capabilities. These scenes are critical for providing realistic, context-rich environments that enable AI/ML and/or autonomous systems to learn, adapt, and perform effectively in real-world, dynamic conditions. A critical component of this effort is the ability to generate dynamic, high-fidelity background scenes that realistically model operational environments. Unlike traditional synthetic data generation, which often focuses on isolated sensor outputs, scene generation must create a coherent, interactive world where autonomous agents can navigate, perceive, and process imagery based on their movement and decision-making.
This presents several challenges. First, scene generation requires accurate modeling of complex environmental factors such as terrain variation, urban structures, vegetation, weather conditions, and electromagnetic propagation all of which impact sensor performance. Additionally, ensuring spatial and temporal consistency across multimodal data (e.g., RF and EO/IR) is far more demanding than simply generating independent synthetic datasets. Autonomous systems rely on their ability to interpret changes in the environment dynamically, requiring realistic physics-based interactions between sensors and the scene. Further, aligning RF and EO/IR perspectives within the same scenario for sensor fusion introduces an added layer of complexity, demanding precise calibration of sensor viewpoints, occlusions, and atmospheric effects.
To accurately model such complex environments, scene generation tools must not only produce synthetic RF and EO/IR data but also ensure that these representations align with real-world sensor measurements. When the underlying environment is well-characterized, scene generation tools can generate multimodal imagery alongside ground truth labels, providing ready-made datasets for AI/ML models and autonomous agents. However, their effectiveness is often constrained by the availability of accurate models and measured databases that capture the necessary radiometric and electromagnetic characteristics of the environment. Addressing these limitations requires the development of software that integrates geospatial data, time-of-day, seasonal variations, measured databases, and land cover data to generate detailed representations of the environment. Furthermore, this software must support standardized scene formats compatible with existing simulation tools such as FLITES (EO/IR) and Xpatch (RF), allowing for flexible resolution and fidelity adjustments based on scenario requirements. Finally, a structured approach should be proposed to refine synthetic scene renderings as real-world measurements become available, improving realism and scene fidelity over time.
PHASE I
Since this is a Direct-to-Phase-II (D2P2) topic, the Government expects that the applicant has demonstrated the ability to automatically build background scenes for either RF or EO/IR simulation tools, such as Xpatch (AFRL/RY) and FLITES (AFRL/RW), using commonly available resources and knowledge of a given geographic region; demonstrated the potential to use background scenes to generate geo-specific, synthetic EO/IR or RF training data for AI/ML models or train/evaluate autonomous systems.
PHASE II
Develop prototype software that ingests geographic coordinates and/or other identifying information on a region-of-interest on Earth, sources representative data on that region, and builds geo-specific background scene data this is compatible with both RF and EO/IR simulation tools. Develop a standardized interface / data format that is compatible with existing RF and EO/IR simulation tools, such as Xpatch and FLITES. Establish and document relevant use-cases. Demonstrate the software's ability to improve AI/ML model performance on geolocation, ATR, scene understanding, and other key tasks in a relevant environment. Plan and coordinate one or more demonstrations to provide proof of concept determination. Perform experiments and analyze results to establish the adequacy of the solution approach and minimize transition risk. Contact potential customers and transition partners to support Phase III activities. Deliver prototype software to AFRL.
PHASE III DUAL USE APPLICATIONS
Add additional classified data sources and work with multiple end-users to provide additional specific capabilities required. Further development will refine the scene generation capability to increase accuracy and fidelity for specific end-users. Commercialize this capability within the DoD and private sectors. Programs of record that would benefit from this effort include but are not limited to FLITES (Fast Line-of-sight Imagery for Targets and Exhaust-plume Signatures), Xpatch (RF signature prediction), JSTAR (Joint Surveillance Target Attack Radar System), Synthetic Scene Generation Model (SSGM), and AFSIM (Advanced Framework for Simulation, Integration, and Modeling). End-users include but not limited to all DoD components.
REFERENCES
Military & Aerospace Electronics. (2024, August 19). Artificial intelligence (AI) and machine learning applications in military operations
Crow, D., Coker, C., & Keen, W. (2006, May). Fast line-of-sight imagery for target and exhaust-plume signatures (FLITES) scene generation program. In Technologies for Synthetic Environments: Hardware-in-the-Loop Testing XI (Vol. 6208, pp. 195-202). SPIE
Hazlett, M., Andersh, D. J., Lee, S. W., Ling, H., & Yu, C. L. (1995, June). XPATCH: a high-frequency electromagnetic scattering prediction code using shooting and bouncing rays. In Targets and Backgrounds: Characterization and Representation (Vol. 2469, pp. 266-275). SPIE.
