Event-Based Infrared Read-Out Integrated Circuit for Neuromorphic Processing

TECH FOCUS AREAS: Microelectronics TECHNOLOGY AREAS: Sensors OBJECTIVE: This topic seeks to model, design, and produce an asynchronous, event-based infrared read out integrated circuit (ROIC) with low power, medium array format, and digital outputs compatible with neuromorphic algorithms. DESCRIPTION: Low power consumption is a persistent goal of military imaging systems. The power consumption is derived from three principal components: operate the imager, process the data, and transmit the image. Infrared imagers in particular must operate at low power levels (less than 500 mW), as power dissipation through the ROIC is more than doubled by the cryogenic cooling requirements, i.e., 1 W of dissipated power in the ROIC will require far more than 1 W of additional cooling capacity by the cryogenic cooler. A strong desire exists to reduce the power consumption in the ROIC and the data processing. Several groups have recently demonstrated low power, event-based sensors in the visible spectrum. These sensors are effectively asynchronous change detection circuits that only send data when the scene changes and produce a basic mapping of where light levels increased and where they decreased. Since a static scene produces no change in the circuit, the data rate and power consumption can be reduced dramatically. Furthermore, neuromorphic processing algorithms have been able to utilize this data directly to perform complicated tasks such as optical flow tracking, automatic target recognition, and stereo imaging. Recent designs have also featured grayscale imaging to enhance the user experience. The combination of both low power operation and processing has the potential to change the imaging paradigm for many systems but has only been demonstrated in the visible spectrum thus far. The goal of this program is (a) to model an event based infrared ROIC in Phase I, (b) to design, develop, and produce the ROIC in Phase II, and (c) to hybridize and demonstrate a full array with neuromorphic processing capabilities in Phase III. The basic requirements for meeting these goals are array formats of 320 x 256 or larger; pixel pitches of 40 microns or smaller; reset times of 10 microseconds or faster; an asynchronous, digital output capable of more than 1E9 events per second; grayscale imaging of 8 bits or greater; and static scene power consumption of 10 mW or less at 120 K. Preference will be given to systems run from commercial infrared camera test dewars with minimal modifications, as well as designs operating using detector material for SWIR (0.9-1.7 m), MWIR (3-5 m), or LWIR (8-12 m). No government-furnished equipment, data, and/or facilities will be provided. PHASE I: Selected Phase I efforts will develop a model for an event based infrared ROIC and imager using provided detector models. The model will be delivered in the form of code (e.g., Matlab, Python) for verification and future validation. PHASE II: Companies selected for Phase II will design and produce an asynchronous ROIC based on the models at the desired array size and pixel pitch. Efforts will demonstrate low power operation under static scenes, as well as high speed operation. Both hybridized focalplane arrays and unhybridized ROICs will be delivered for testing, with supporting hardware interface control documentation and control software. PHASE III DUAL USE APPLICATIONS: Phase III efforts will demonstrate a fully packaged camera with a neuromorphic processing chip. Spiking neural network device technology is preferable. Efforts will leverage emerging event-based sensing algorithms to demonstrate. NOTES: The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the proposed tasks intended for accomplishment by the FN(s) in accordance with section 5.4.c.(8) of the Announcement and within the AF Component-specific instructions. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws. Please direct questions to the Air Force SBIR/STTR Help Desk: usaf.team@afsbirsttr.us REFERENCES: 1. Delbr ck, T., et al., Activity-Driven, Event-Based Vision Sensors. Proceedings of 2010 IEEE International Symposium on Circuits and Systems, 2010. p. 2426; 2. Cohen, G.K., et al., Spatial and Temporal Downsampling in Event-Based Visual Classification. IEEE Transactions on Neural Networks and Learning Systems, 2018. Vol. 99 p. 1; 3. Benosman, R., et al., Event-based visual flow. IEEE transactions on neural networks and learning systems, 2014. Vol. 25 p. 407. 4. Event-based Vision - A Survey - arXiv: 1904.08405v3 [cs.CV] 8 Aug 2020; IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020; 5. A 1280 720 Back-Illuminated Stacked Temporal Contrast Event-Based Vision Sensor with 4.86 m Pixels, 1.066GEPS Readout, Programmable Event-Rate Controller and Compressive Data-Formatting Pipeline; 2020 IEEE International Solid- State Circuits Conference - (ISSCC) KEYWORDS: Neuromorphic imaging; event-based imager; read out integrated circuit; ROIC; infrared detector; infrared camera; asynchronous time-based image sensor