2319321

NCS-FR: Engineering brain circuits for complex scene analysis - everyday social situations, like a crowded party, a restaurant, a classroom, or an open-plan workplace, involve multiple speakers and listeners and the hum of background noise.

In these complex sound environments, humans with typical hearing are able to identify and listen to individual sound sources, for example what a single speaker is saying, while ignoring the other sound sources, for example someone else's phone call or background noise, like cars driving down the street.

This is an example of a general problem called complex scene analysis (CSA), and a full understanding of how humans with typical hearing solve this problem has remained elusive to scientists from a diverse range of fields - neuroscience, computer science, speech recognition, and engineering - even after more than 50 years of research.

Because of this, CSA remains a problem for many humans, like those with hearing impairment, for medical devices, like hearing aids, and for technology, for example automatic speech recognition systems.

This project investigates the neural basis of complex scene analysis in typical hearing, and, based on these discoveries, develops a brain-inspired algorithm for CSA. This project will ultimately improve quality of life through a variety of applications, for example for improving the effectiveness of hearing aids and speech recognition technologies.

Solving this problem requires an interdisciplinary effort, and as part of the research, an educational platform is developed to train students to integrate knowledge from a variety of disciplines that makes them better able to address challenging and important societal problems.

This project integrates three interdisciplinary research threads to develop the brain-inspired algorithm. The first thread uses brain imaging in humans performing CSA with an integrated wearable device that measures brain signals (functional near-infrared spectroscopy and electroencephalography), and machine learning methods to decode where a subject is attending in a complex audiovisual scene.

The second thread investigates cortical circuits for CSA in attentive states, which are thought to enhance CSA performance. This thread integrates electrophysiology, optogenetics, behavior, and computational modeling in mice, a model system with well-established, powerful experimental tools for unraveling cortical circuits.

The third thread designs an attention-steered algorithm for the wearable device that selectively processes an attended source in a complex scene, integrating the attended location decoded from a subject's brain signals (thread 1), and a model of cortical circuits in attentive states (thread 2). This thread optimizes the algorithm to generate a fast, compact, energy efficient, and state-of-the-art algorithm for CSA and evaluate its performance in humans.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Trustees Of Boston University

THE GOAL OF THIS FUNDING OPPORTUNITY, "INTEGRATIVE STRATEGIES FOR UNDERSTANDING NEURAL AND COGNITIVE SYSTEMS", IS IDENTIFIED IN THE LINK: HTTPS://WWW.NSF.GOV/PUBLICATIONS/PUB_SUMM.JSP?ODS_KEY=NSF21517

47.084 - NSF Technology, Innovation, and Partnerships

NSF Office of Multidisciplinary Activities (OMA) [NSF]

National Science Foundation (NSF)

Boston, Massachusetts 02215-1703 United States

Single Zip Code

Integrative Strategies for Understanding Neural and Cognitive Systems (21-517)

Amendment Since initial award the End Date has been extended from 08/31/28 to 12/31/28 and the total obligations have increased 116% from $1,646,782 to $3,561,895.