GigEVision-compliant Event-based Cameras

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Microelectronics; Space Technology; Trusted AI and Autonomy OBJECTIVE: Develop camera interface hardware and software, which allows commercial off the shelf (COTS) event based sensors to utilize standard machine vision interfaces such as Ethernet hardware connections and GigEVision or GenICam/GenTL software. DESCRIPTION: Event based sensing is a novel modality for capturing transient data objects in a scene. Born of the desire to mimic the way biological sensing apparatus operate (i.e., the human eye), event based cameras (EBCs) are fundamentally different than standard frame cameras, and therefore the resulting data stream is unique. EBC data must be read out, processed, and interpreted differently than how standard imagery is done today. To date, there does not exist a standard interface, in either hardware or software, by which the data stream is extracted from the EBC. This causes great difficulty for government researchers in operating, testing, and utilizing EBC capabilities. The objective of this SBIR topic is to develop a new EBC with the hardware and software needed to accept and connect the EBC and its data to other government imaging and data processing apparatus. The prototype EBC is required to use COTS sensor components. The electronics, boards, physical connections, and software will be developed within this effort. Having standard hardware and software interfaces that can be appended to available COTS event based sensor components will enable this unique technology to be tested, integrated, and used at government facilities, empowering government development and unlocking the capability for the warfighter. Other manufacturing applications to this technology is for manufacturing capabilities in the DoD industrial base such as Shipyards and other locations for material prep, inventory, etc. General Requirements and Specifications: 1) The software interface must comply with Motion Imagery Standards Board requirements. 2) The software interface must operate with low latency (< 2ms per payload). 3) The software interface must comply with GigEVision or GenICam/GenTL software standards. 4) The new EBC must be configurable via the GenICam interface. 5) The hardware interface must utilize 10 Gigabit Ethernet (10GbE/10GigE) technology for transmitting data and configuring the EBC. 6) The new EBC core must leverage COTS event-based sensor components. 7) The new EBC should have a small form factor (~4 x4 x4 ) and have integrated digital-to-analog converters (DACs)/references for the on-chip event based sensor component biases. 8) The new EBC should have an external general purpose input/output (GPIO) interface which can be used for synchronization and timing. 9) The new EBC should utilize a field-programmable gate array (FPGA) or similar as the intermediary between the event based sensor components and the 10GigE interface. This device should be comparable to the specification of an Advanced Micro Devices (AMD) Kintex-7 and leverage 2MB of external memory for potential frame buffer. PHASE I: Develop concepts and schematics for electrical and mechanical components of a new EBC and 10GigE physical interface which is based upon COTS event based sensor components and will allow data transfer via Ethernet cables. Develop concepts and a block diagram of a software package which can read in data streams from this new EBC, configures the EBC biasing/readout modes and complies with GigEVision or GenICam/GenTL software standards. Demonstrate the feasibility of the concepts in meeting Navy and Naval Enterprise needs and establish the concepts for development into a useful product. Establish feasibility through material testing and analytical modeling, as appropriate. Provide a Phase II development plan with performance goals and key technical milestones and that addresses technical risk reduction. PHASE II: Develop a prototype for evaluation. Evaluate the prototype to determine its capability in meeting the performance goals defined in the Phase II development plan and the Navy requirements for the hardware and software interfaces. Support Navy requirements for any testing, such as submittal of Navy Cybersecurity Waiver Board interaction, submissions and approvals and development of a system security plan. Demonstrate system performance through prototype evaluation and modeling or analytical methods over the required range of parameters. Use evaluation results to refine the prototype into an initial design that will meet the Naval Enterprise requirements. Prepare a Phase III development plan to transition the technology to Navy enterprise use. PHASE III DUAL USE APPLICATIONS: Support the Navy with putting this product into useful service in government facilities. This product could be leveraged by commercial camera producers who develop, manufacture, and sell EBCs. Possible transition to Tech candidate or Future Naval Capabilities (FNC) or Innovative Naval Prototype (INP) for a Program of Record. REFERENCES: Motion Imagery Standards Board (MISB) Geospatial-Intelligence Standards Working Group. https://gwg.nga.mil/gwg/focus-groups/Motion_Imagery_Standards_Board_(MISB).html GenICam. European Machine Vision Association. https://www.emva.org/standards-technology/genicam/ GigE Vision . https://www.automate.org/vision/vision-standards/vision-standards-gige-vision Event Based Vision Sensors. | FRAMOS. https://www.framos.com/en/products-solutions/image-sensors/technologies/event-based-sensors KEYWORDS: Event based cameras; camera interface; software interface; Motion Imagery Standards Board; MISB standard; Ethernet; hardware interface; automation; machine learning