TECH FOCUS AREAS: Cybersecurity TECHNOLOGY AREAS: Information Systems OBJECTIVE: This topic's objective is to develop, demonstrate and transition analytics to detect, classify and forecast the impact of information maneuvers (based on BEND information maneuvers framework) based on visual content (video, memes and pictures) and audio chat in order to support meaning making and decision-making regarding influence, course of action assessment and forecasting of behaviors, events. No BEND analytics currently exist for visual content. In fact, limited analytics for visual content exist at all -- they largely address activity detection, actor detection (akin to "entity detection" or "event detection") or characterization of affect. Likewise, there are not BEND (influence characterization) analytics for audio chat, as this is an emerging online discourse space. Current audio-based analytics focused on content, role taking within conversations ("politeness", etc.) could be leveraged to develop new analytics to characterize influence in this domain. DESCRIPTION: Operating in the information environment today is highly challenging for military warfighters. Due to the difficulty in assessing influence in social media, a Social Digital Media playbook was developed to help characterize actions, events, and communications (Beskow and Carley, 2019). Specifically, the BEND framework accounts for how social media algorithms interact with different user activities. There are sixteen simple information maneuvers with half being community maneuvers and half being content maneuvers. Research is needed to extract the maneuvers from visual content (e.g., videos, memes) and also to better visualize the maneuvers. Additionally, research is needed on social media platforms beyond Twitter and with emerging platforms and capabilities such as voice chat (e.g., clubhouse). New analytics can assist planners, information operators, intelligence analysts with adaptive planning, triggering and cueing of sensors, strategic communication, etc. Further, by understanding the online behaviors and mechanisms of influence, it can help forecast behaviors offline (e.g., civil unrest leading to protests) which could lead to better military intervention strategies. Ideally, the approaches will include multiple parameter spaces to control for various knowledge topics, different events, varying network sizes, and different actors (including bots). Capabilities including non-English speaking contexts would particularly align with this topic. Approaches solely focused on disinformation (e.g., fake news, deepfakes) do not align with this topic. No government furnished materials, equipment, data, or facilities will be provided. PHASE I: Develop software (analytic algorithms, models, and visualization) for characterizing the 16 BEND information maneuvers that are visual (e.g., images, memes, videos) and / or voice (e.g., voice chat). Proof of concept demonstration of the software for detection and classification of 4 BEND maneuvers from each type (content -- positive, negative, network -- positive, negative) across multiple types of visual content (videos, memes, pictures) and/or audio chat (e.g., Clubhouse). Deliverables are detection, classification, visualization software as well as full documentation of algorithms and characterization of algorithms (Receiver Operating Characteristic curves) for detection and classification software and narrative addressing proposed approach for expanding to all 16 BEND maneuvers in final report. PHASE II: Companies selected for Phase II will apply the knowledge gained in Phase I to mature and integrate analytics and to further develop the interface, capabilities and training components needed to make the technologies transition to military customers, marketing, etc. Expand and develop the model to cope with real-time information flows and evolving information tactics. Demonstration of detection, classification of all (16) BEND maneuvers, including detection, classification of associated maneuvers in campaign (e.g., BOOST and BUILD maneuvers) in visual content (videos, memes, pictures) and/or audio chat. Anticipate/forecast impact on target audiences. Deliverables are software for detection, classification, visualization of both individual BEND maneuvers as well as associated maneuvers as well as a final report with full documentation of algorithms and characterization of algorithms (Receiver Operating Characteristic curves) for detection and classification software and narrative addressing proposed approach for visualization of influence campaigns, operationalization/transition to customers, including drill-down, supporting information to support meaning making by operators and anticipated/forecasted impact on target audiences. Additional deliverable is a software test dataset to be used to demonstrate the software/visualization to customers. PHASE III DUAL USE APPLICATIONS: Phase IIIs will apply the knowledge gained in Phase II to further develop the interface, capabilities and training components needed to make the technologies transition to military customers, marketing, etc. They will expand and develop the model to cope with real-time information flows and evolving information tactics. Efforts will demonstrate ability to detect, classification of BEND maneuvers, including a campaign (associated BEND maneuvers) in visual content (videos, memes, pictures) and audio chat. Further, they will demonstrate capability for users to visualize maneuvers, drill down to data, and forecast impacts on target audiences, including the impacts of counter maneuvers. REFERENCES: Beskow, D.M. and Carley, K.M., Social Cybersecurity: An Emerging National Security Requirement, Military Review Army University Press March-April 2019, 125 (2019) https://www.armyupress.army.mil/Journals/Military-Review/English-Edition-Archives/Mar-Apr-2019/117-Cybersecurity/ Nimmo, B., Hubert, I. and Cheng, Y., Spamouflage Breakout: Chinese Spam Network Finally Begins to Gain Some Traction, Graphika website, https://public-assets.graphika.com/reports/graphika_report_spamouflage_breakout.pdf; Erol, R., Rejeleene, R., Young, R., Marcoux, T., Hussain, M.N., and Agarwal, N., YouTube Video Characterization Using Moviebarcode, Proceedings HUSO 2020: The Sixth International Conference on Human and Social Analytics, 15-19; Fenstermacher, L. and Larson, K., Multi-Source Insights for Discernment of Competition Threat, Proceedings Signal Processing, Sensor/Information Fusion and Target Recognition XXIX, 26-30 April 2020, Anaheim, CA; Kurutz, S. (2021, Feburary 20). Join Clubhouse! Umm, What Is Clubhouse? The New York Times. https://www.nytimes.com/2021/02/20/at-home/clubhouse-app-explainer.html KEYWORDS: social-cyber data; social media analytics; network analytics; social media visualization; audio chat; voice chat; forecasting; classification; influence; actors; communities; networks; maneuvers; information maneuvers; influence; information operations; deep fakes; bots; information maneuver
