TECHNICAL POINT OF CONTACT (TPOC)
Matthias Banet
Patricia Hall
PROJECTED CMMC LEVEL REQUIREMENT
Level 2 (Self)
TECHNOLOGY AREAS
Air Platform
|
Sensors
MODERNIZATION PRIORITIES
Advanced Computing and Software
|
Directed Energy (DE)
|
Trusted AI and Autonomy
KEYWORDS
3D imaging, Aimpoint Maintenance, Coherent optical sensing; Deep turbulence; Anisoplanatic turbulence; Adaptive optics; Beam control; Artificial Intelligence; Image reconstruction
OBJECTIVE
To develop a real-time computational pipeline that meets the demanding latency and throughput requirements for real-time enhanced accurate 3D imaging for directed energy applications.Motivation: The growing threat of drone-based attacks has underscored the critical need for advanced counter-drone technology in the DoD. One area of interest is the potential for 3D imaging modalities to outperform 2D modalities in these circumstances. 2D imaging modalities cannot explicitly separate an object from clutter, for example, without additional image processing/hardware. Moreover, the use of coherent illuminators provides 2D imagery with non-uniform illumination patterns (scintillation) on the target and speckle noise in the target imagery. These shortcomings of 2D imaging & tracking are especially relevant to the fine track segment of the directed energy (DE) kill chain. As such, we wish to investigate 3D imaging (a.k.a. LiDaR or ladar) in conjunction with machine-learning-based declutter and image reconstruction approaches to fill in the technology gaps that are present for 2D imaging and tracking. While this topic is focused on counter-drone technologies, a top priority for the Office of the Secretary of Defense, this technology is relevant for missile defense in general, including ballistic, cruise, and hypersonic missiles, which were referenced in a recent executive order [1].Benefit: AFRL/RD is interested in investigating solutions that address the problems of background clutter, scintillation, and speckle in DE-specific target tracking engagements. 3D imaging provides a leg-up over 2D imaging for several key reasons: (1) 3D imaging allows for precise range gating of the target signature from cluttered backgrounds, (2) range images, which are generated from the 3D data itself, are extremely robust against the deleterious effects of scintillation and speckle, and (3) 3D coherent imaging allows for the correction of phase aberrations that arise due to aero-optics and turbulence. Providing 3D imaging & tracking capabilities to the warfighter will greatly enhance counter-drone and counter-missile defenses as well as increase intelligence, surveillance, and reconnaissance capabilities as an added benefit.Main Goal: To develop a 3D imaging & tracking prototype that meets the demanding latency and throughput requirements for real-time enhanced accurate 3D imaging for DE applications.Subgoals: 1. (Basic Research) Study the pros and cons of 3D imaging modalities and determine system requirements for engagements that are of interest to AFRL/RDL. Design 3D imaging system.2.(Applied Research) Develop and improve 3D tracking and aimpoint maintenance algorithms to be used for 3D data in general.3.(Applied Research) Develop 3D imaging prototype by leveraging the results of Tasks 1 & 2into hardware development.Deliverables: Reports on PowerPoint and text-based manuscript formats. Tracking algorithms and software for 3D image processing. Hardware prototype of 3D imaging system (Phase II)
DESCRIPTION
Current tracking capabilities in DE utilize 2D imagery. Low flying targets against cluttered backgrounds (e.g., drones) present a challenging tracking environment for track algorithms that leverage 2D information alone. A current tracking algorithm might use something like a Fitts correlation tracker [2], which correlates features in the scene with a known template of the target. This requires target identification followed by reference to a look-up table to determine the proper template to use with the correlation tracker, which can be difficult with 2D imagery, as a 2D image of the target will only provide a projection of the target image along the optical axis. 3D coherent imaging with digital holography [3,4] provides a solution for the problems associated with 2D imaging that are described here. Emerging coherent 3D imaging modalities provide complex-valued images (referred to as 2.5D images) with amplitude corresponding to a conventional 2D image and phase corresponding to the range to the target. The range images themselves (which describe the range to the target for each transverse pixel) are highly robust to scintillation and speckle [5,6]. Going further, novel methods in image restoration that include a combination of physics-based [7-11] and machine-learning [10,11] models can provide effective and computationally efficient ways to remove the anisoplanatic distortions in the image. Note that alternative 3D imaging methods do exist (e.g., Geiger mode 3D imaging with avalanche photodiodes [5]). The Phase I effort should include an analysis of existing methods before selecting one to explore for the given application. The topic proposed here is nominally focused on the fine track portion of the DE kill chain, though the technology could be used for acquisition/coarse tracking as well. We briefly summarize the key features of fine tracking in DE, which performers should keep in mind when designing systems: For fine tracking, the atmosphere and aero-optic disturbances impose phase aberrations that can severely degrade imagery. Importantly, tracking at this stage influences aimpoint maintenance, which requires fine track loops to run at much higher framerates (> 1 kHz) to keep up with the changing atmosphere while employing narrower fields of view (fractions of a degree). 3D imaging provides a path forward for enhanced pose estimation and aimpoint maintenance in the context of fine tracking. With regards to illumination for fine tracking, active coherent illumination can result in non-uniform illumination patterns on target, due to scintillation, and in speckle noise in received target imagery. Partially coherent (or multiple coherent) illuminators can reduce the effects of these two noise terms, but at the cost of the ability to perform coherent imaging and/or additional illuminators. On a similar note, coherent imaging techniques (such as digital holography and phase retrieval) can allow for digital image correction of anisoplanatic phase errors using physics-based [7-11] and machine-learning-based [10,11] image reconstruction algorithms. Note that active illumination, in general, enables tracking technology that can perform in day or night. In general, performers should be aware of the general requirements for fine tracking and should tailor designs to the technology best suited for the application. This topic seeks the development and testing of a computationally efficient processing pipeline that meets the requirements for real-time integration into a directed energy system. The goals are to design (Phase I) and demonstrate (Phase II) the processing pipeline for a 3D imaging system that can operate effectively over long standoff ranges through aerooptics/turbulence with extended, non-cooperative targets against cluttered backgrounds. The Phase I effort will design the 3D measurement technique, corresponding algorithms, and a concept for the integrated hardware. This should include analysis of the latency and throughput of the algorithms and processing hardware for real-time, closed-loop aimpoint maintenance. Then, the Phase II effort will implement the approach in a prototype demonstration system, which would deploy such algorithms onto high-speed processors and integrate with hardware that allows for real-time closed-loop correction of the line-of-sight. Such a demonstration could take place with a scaled-laboratory optical system. Alternatively, an outdoor demonstration would be appropriate as well. Key products include: 1) algorithms and software to improve 3D imagery and maintain an aimpoint by taking advantage of the 3D data, and 2) demonstrated prototype processor with real-time test results.To meet the objective, potential tasks in several research directions are described below. These tasks are open ended, and comprehensive results are not expected before Phase III.Basic research in 3D imaging1.Determine system-level requirements for a real-time 3D imaging and tracking system2.Explore the available motion-compensated 3D imaging techniques and select one for the prototype system after a pros/cons analysis3.Develop an empirical and/or theoretical signal-to-noise ratio model of the 3D imaging systemin questionApplied research in 3D tracking and aimpoint maintenance1.Mature and develop existing 3D tracking algorithms to work with 3D data from a vast array of imaging modalities2.Determine what benefits arise from tracking algorithms that make use of coherent 3D images (and 2.5D images as well), in specific3.Develop software architecture that is capable of real-time trackingHardware development (Phase II and beyond)1.Leverage system-level requirements (see Task 1) to build a prototype illuminator system2.Leverage software development (see Task 2) to integrate real-time tracking capabilities3.Demonstrate hardware in laboratory or outdoor testing environment
PHASE I
Select a 3D coherent measurement technique and design a method for accurate estimation of target pose and aimpoints in long-range DE systems. 2. Analyze the feasibility of combined software and hardware pipeline for meeting real-time, closed-loop aimpoint maintenance needs. 3. Develop initial design and hardware specifications to build a prototype system in Phase II TECHNICAL OUTCOMES: Design and hardware specifications of 3D imaging system for construction of prototype in subsequent phase. Basic research results on pros and cons of various 3D imaging systems
PHASE II
Build a processor prototype system that is capable of interfacing with real-time, closed-loop hardware for DE aimpoint maintenance. 2. Demonstrate the system in a scaled-laboratory optical system or outdoor test range TECHNICAL OUTCOMES: Laboratory system capable of testing 3D imaging modalities of interest and/or tracking algorithms of interest.
PHASE III DUAL USE APPLICATIONS
Complete real-time computational pipeline that meets the demanding latency and throughput requirements for real-time enhanced accurate 3D imaging for directed energy applications. 2. Conduct experiments with real-time hardware and existing beam director technology (e.g., the Beam-Control, Targeting Resource Advanced Integration Lab) TECHNICAL OUTCOMES: Demonstration of coherent 3D imaging as a feasible DoD tracking asset for counter-drone kill chains
REFERENCES
D. J. Trump "Executive Order on The Iron Dome for America" (2025 Jan 27). J. Fitts, Precision correlation tracking via optimal weighting functions, 18th IEEE Conference on Decision and Control, 280-283, (1973). B.W. Krause, Motion compensated multi-wavelength digital holography, US Patent 9,581,967 (2017 Feb 28). M. T. Banet, J. R. Fienup, Speckle decorrelation effects on motion-compensated, multi-wavelength 3D digital holography: theory and simulations, Opt. Eng. 62(7) 073103 (2023). J. Ogren, 3D Tracking Demonstration System Testing, 2024 JDETO Technical Review (2024). M. T. Banet and J. R. Fienup, Image sharpening on 3D intensity data in deep turbulence with scintillated illumination, Proc. SPIE 118360E, (2021). S. T. Thurman and J. R. Fienup, Phase-error correction in digital holography, J. Opt. Am. Soc. A 25, 983 994 (2008). S. T. Thurman and J. R. Fienup, Correction of anisoplanatic phase errors in digital holography, J. Opt. Soc. Am. A 25, 995 999 (2008). M. T. Banet, J. R. Fienup, J. D. Schmidt, and M. F. Spencer, 3D multi-plane sharpness metric maximization with variable corrective phase screens, Appl. Opt. 60, G243 G252 (2021). C. J. Pellizzari, M. F. Spencer, and C. A. Bouman, Imaging through distributed volume aberrations using single-shot digital holography, J. Opt. Soc. Am. A 36, A20 A33 (2019).
C. J. Pellizzari, et al., Coherent plug-and-play artifact removal: Physics-based deep learning for imaging through aberrations. Optics and Lasers in Engineering 164(2023).
QUESTIONS & ANSWERS
NOTE:
To ask a question, you must
log in or create an account
for the DSIP.
