Improved Road Network Extraction Through Reinforcement Learning

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Trusted AI and Autonomy OBJECTIVE: Develop the next generation automated geographic feature extraction by replicating human performance through reinforcement learning. DESCRIPTION: The National Geospatial-Intelligence Agency (NGA) has a requirement to generate foundation feature data vector (FFD) features like roads and building outlines. Today's FFD production process remains a labor intensive, time consuming manual process, and slow to deliver products that rapidly become outdated. NGA's traditional production, evaluation, and dissemination cycle takes too long to allow for regularly updated feature data. Methods for automated feature extraction (AFE) have advanced significantly (Chen, et al., 2022). Most of these computer vision AFE methods rely on convolutional neural network (CNN) interpretation and segmentation of satellite imagery. The reliance on visual interpretation leaves these methods susceptible to incomplete, inaccurate, and non-routable networks because of obstructions (e.g. tree-lined roads) visible in the source imagery CNN-based AFE cannot properly interpret. This topic will develop new hybrid artificial intelligence (AI) methods to augment computer vision results with reinforcement learning to replicate human decision making. Reinforcement Learning (RL) is a technique in AI for teaching machines to make decisions (Sutton & Barto, 2018) that has been applied to classic video games like Pac Man (DeNero, Klein, & Abbeel, 2023). RL's central advantage is that it learns sequences of actions similar to the way humans map, such as generating points along a roadway or the side of a structure, to facilitate validation. New techniques for feature tracing similar to how humans would approach digitizing should significantly increase vector data production and accuracy from imagery. RL has been further applied to automated and semi-automated feature extraction to refine pixel-wise segmented vector data directly (Botteghi, Sirmacek, & Unsalan, 2021). Proposals for this topic should address a similar hybrid approach for new methods to enhance or replace deep learning/computer vison only approaches, augmented with decision making to improve completeness. The method should scale to city or regional scales across diverse geographic context and environments. NGA expects that software prototyping/development under Phase II will be subject to export controls as described under ECCN 0D521 in the 0Y521 series table found in Supplement No. 5 to part 774 of the Export Administration Regulations (EAR). Phase II work most likely be conducted on NIST SP 800-171 compliant information systems. PHASE I: The Phase I proposal should focus on demonstrating the feasibility of extracting roads in at least one small geographic area using an RL approach. Phase I will also provide a concept of operations to build and deploy at larger scales with associated costs. PHASE II: Phase II will focus on developing a mature capability creating two (2) prototypes, one for roads and a second expanding to building footprint extraction, both in geographic areas of varying complexity (e.g. urban and rural, canopy vs. desert). Phase II will also test the accuracy improvements over existing computer vision-only methods. PHASE III DUAL USE APPLICATIONS: Fully develop and transition the technology and methodology based on the research and development results developed during Phase II for DOD and other commercial feature extraction applications. REFERENCES: 1. Botteghi, N., Sirmacek, B., & Unsalan, C. (2021). RLSNAKE: A Hybrid Reinforcement Learning Approch for Road Detection. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B3-2021. doi:https://doi.org/10.5194/isprs-archives-XLIII-B3-2021-39-2021; 2. Chen, Z., Deng, L., Luo, Y., Li, D., Marcato Junior, J., Nunes Goncalves, W., . . . Li, D. (2022). Road extraction in remote sensing data: A survey. International Journal of Applied Earth Observations and Geoinformation, 112. doi:https://doi.org/10.1016/j.jag.2022.1028333. 3. DeNero, J., Klein, D., & Abbeel, P. (2023, May 22). UC Berkeley CS188 Intro to AI. Retrieved from http://ai.berkeley.edu/project_overview.html; KEYWORDS: Reinforcement Learning; Artificial Intelligence (AI); Satellite Imagery; Automated Feature Extraction (AFE)