RT&L FOCUS AREA(S): Artificial Intelligence/ Machine Learning; Autonomy TECHNOLOGY AREA(S): Electronics; Sensors The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with section 3.5 of the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws. OBJECTIVE: DTRA seeks the capability for its robotic systems, developed to explore and exploit, in GPS denied and communications limited environments, to learn from each other. DESCRIPTION: Under the Modular Autonomous CWMD Systems (MACS) program, among others, DTRA is developing robotic and autonomous systems (RAS) to map, explore, and characterize subterranean facilities such as tunnels, caves, urban underground, and military purpose bunkers. These facilities can be extensive and include multiple levels, elevation changes, obstacles, dim and variable lighting, and other challenging conditions. In addition, GPS is unavailable in such locations and communications between users and RAS, and amongst RAS platforms, is limited in bandwidth and range. The primary mission of these RAS is to explore, map, and catalog in these environments. However, autonomous resupply, payload delivery, network forming, and other mission scenarios should be considered. The environment and missions dictate that various types of robotic platforms are necessary and the communications challenge dictates that a high degree of autonomy is necessary on each platform with the associated sensors and data load. The communication challenge also limits the amount of data that may be passed from platform - to - platform and platform - to user which makes application of a centralized learning concept less feasible. Distributed learning solutions, such as combinations of federated learning, transfer learning, and/or distributed multi-agent reinforcement learning are approaches that enable model training on a large amount of decentralized data. That is, they enable the model to be passed over the network rather than the data. This could be extremely beneficial to the underground exploration mission of DTRA if telemetry, health and status, map and environmental information (such as air flow, air quality, lighting, traction), as well as mission specific information such as the presence or concentration of certain chemicals or radiological information, could be encoded into a model that could be more easily distributed among platforms and users that have only intermittent connectivity. This could allow platforms to more quickly navigate in areas that have already been explored by other platforms and allow more effective decisioning by each platform individually. PHASE I: Design and develop both the models and a learning architecture for robotic platforms exploring and cataloging a subterranean environment such as a tunnel or cave. In simulations of subterranean areas with communication challenges, demonstrate the capability for multiple platforms to be controlled by humans and to learn from that control as to what are obstacles, what is of interest, what is not, etc. Locally train models on the data being collected and then update the models and for other platforms to receive and benefit from those updated models. Similarly, show platforms learning from each other by locally training models and then updating the shared model. A final demonstration should show at least more efficient path planning being developed by platforms that receive an updated model after a different platform has already explored an area. PHASE II: Continue to develop the learning system and adapt it to the DTRA mission and DTRA platforms for incorporation into the group. Incorporate decisioning into the models that mitigates or adds information to the environmental conditions determined. Develop a tasking capability such that one platform may task other specialized platforms. Tasking may be dependent on specific object recognition or some other queue. Specializations may include carrying additional lighting, communications repeaters, CBRNE sensors, or other. Demonstrate the capability for the platforms to learn about the environment as well as the capabilities and limitations of the other systems in the group. A penultimate simulation demonstration should show autonomous tasking and path finding. For example, a UAV may be tasked to find the best route for a UGV to take given a particular set of obstacles and a system with specialized equipment will be called upon as required. Additionally, develop and simulate a small scale demonstration to be performed on actual hardware. Determine the hardware requirements and identify, develop, or otherwise procure it and perform a small scale demonstration in the same or similar environment that was included in the simulation. PHASE III DUAL USE APPLICATIONS: Continue to develop and refine the Phase II product into a useful asset for DTRA. Adapt the product application for DTRA specific testing to include development or application of safety and security measures as required. REFERENCES: 1. Xiao, Y., Hoffman, J., Xia, T., and Amato, C. Learning Multi-Robot Decentralized Macro-Action-Based Policies via a Centralized Q-Net. https://arxiv.org/abs/1909.08776. 2020.; 2. G. Sartoretti, W. Paivine, Y. Shi, Y. Wu and H. Choset, "Distributed Learning of Decentralized Control Policies for Articulated Mobile Robots," in IEEE Transactions on Robotics, vol. 35, no. 5, pp. 1109-1122, Oct. 2019, doi: 10.1109/TRO.2019.2922493.; 3. Taylor, Adam & Dusparic, Ivana & Gu riau, Maxime & Clarke, Siobh n. (2019). Parallel Transfer Learning in Multi-Agent Systems: What, when and how to transfer?. 10.1109/IJCNN.2019.8851784.; 4. Rieke, Nicola. 2019. What is Federated Learning?https://blogs.nvidia.com/blog/2019/10/13/what-is-federated-learning/; 5. Bhattacharya, Santanu. 2019. The New Dawn of AI: Federated Learning. https://towardsdatascience.com/the-new-dawn-of-ai-federated-learning-8ccd9ed7fc3a;
