TECHNOLOGY AREAS: Information Systems OBJECTIVE: The objective is to develop, demonstrate, and transition LLM-enabled software for the assessment, characterization, and forecasting of cognitive warfare and information operations impacts on groups, communities, or populations. This capability will include a user interface that permits a customer to provide prompts and inquiries that run analyses, present assessments, and generate potential courses of action or expected outcomes. The software is to pull quantitative and qualitative data on visual, auditory, or textual information from multiple vectors of media to support sense-making and decision-making for operations in the information environment. An important aspect of this effort is in developing the ability present an emic (1st person) perspective of the target populations, with a focus on sentiment analysis correlated to regional events, actors, and messages that can code contextualized emotions and stances/opinions based on the language found in the data. Thus, a psychographic model of a local population can be crafted, visualized, and depicted in time and space. DESCRIPTION: Competing in the information environment is highly challenging given the large amount of information that must be analyzed to ascertain whether opportunities or threats exist at a particular time and in a particular informational space. Concurrent with this is the constant need for data sharing to better coordinate efforts across diverse entities. Forecasting, detecting, and measuring the effectiveness of information maneuvers or employed cognitive warfare tactics is difficult. All of this is complicated by the high speed at which information flows, outpacing the human's capacity for processing and planning. To maintain situational awareness and support sense-making or forecasting in order to achieve information mastery, something beyond mere analytics is needed. Something akin to an automated information wingman might fill that gap, which can provide multiple perspectives, including the emic (1st person) perspective grounded in social science, and a multi-scale (person, culture, region, country) lens to contextualize and assess information maneuvers. This solution will contrast sharply with many current approaches that integrate scraping software and AI-enabled technology that present information and assessments from the etic (who, what, where) perspective that focus on the facts or events and not the subjective character of a people. New analytics, modeling, and visualization capabilities can assist planners, information operators, and intelligence analysts with influence campaign planning, strategic communication, and assessments. This capability will support military operations during conflict and in deterrence efforts or strategic competition. Uses of generated group, community, or population psychographic models can include operations, activities, and investments supporting real-world combatant commander objectives, or can include notional data and simulations for wargaming and the development of tactics, techniques, and procedures related to cognitive warfare. These generated models could be depicted in a variety of ways using AI data-visualization techniques (e.g. chatbot directed analysis) as well as AI-simulation techniques. By understanding the contextualized sentiments and emic perspectives of a target audience, military (or market) intervention strategies can be better devised. Ideally, the provided approaches will include multiple parameters to control for and integrate various knowledge topics, discrete events, varying community or network sizes, different languages/cultures, and diverse actors and influencers in the information environment. Approaches solely focused on disinformation do not align with this topic, and though this could be a component, this capability would concern reaction and response to legitimate events and potentially actionable activities done in response to them. No government furnished materials, equipment, data, or facilities will be provided. PHASE I: Develop and demonstrate an LLM, AI-enabled software (analytic algorithms, models, visualizations) that pulls quantitative and qualitative data from visual, auditory, or textual information via multiple vectors of media to support sense-making and decision-making for operations in the information environment. Specifically, one that provides the ability to present an emic (1st person) perspective of target populations, with a focus on contextualized sentiment analysis correlated to regional events, actors, and messages that codes emotions and stances/opinion/viewpoints related to them based on the language found in the data. Information characterization should include multi-media visual (images, memes, videos), auditory (voice chat, audio recording, music), or textual (articles, tweets) content. A proof of concept demonstration would include a software capable of collection, analysis, and assessment of real or synthetic data, integrating multiple media vectors and multiple types of informational content to characterize a multi-facet population model. The demonstration of disaggregated sentiment detection/collection, encoded with key language indicators and bounded by regional area is key to this phase, using notional or real-world data. A feasibility study is to be conducted, comprising the use of research tools (e.g. MTurk) in a pilot experiment that can be presented to operational customers, validating the models and evaluating if the perception of the user interface during demonstration affects performance. A lexicon of indicators is to be delivered, embedded into the UI or as a separate product to ensure that analysts assess its content in the most accurate and consistent way. Other deliverables include a dataset, report outlining the assessment/modeling technique, algorithms, generative AI approach, and visualization software or dashboard that showcases the population model with various streams of information. Customers for the capability should be identified during this phase. PHASE II: Companies selected for Phase II will apply the knowledge gained in Phase I to mature and integrate analytics and to further develop the interface, capabilities, and components needed to make the technologies transition to military customers and the open market. It will expand and develop the model to cope with real-time information flows and evolving information tactics. Deliverables are collection, assessment/modeling, and visualization software with the integration of an artificial general intelligence that uses regional media outlets and / or social media of visual, audio, or text-based data to characterize the information environment. Deliverables also include a final report with full documentation of algorithms and LLMs for detection/collection, modeling, and depiction software. An additional deliverable is a comprehensive software test dataset (synthetic or real-world) to be used to demonstrate the software/visualization to customers. PHASE III DUAL USE APPLICATIONS: Phase III selectees will apply the knowledge gained in Phase II to further develop and complete the interface, capabilities and components needed to make the technologies transition to military customers and the open market. It will expand and develop the model to cope with real-time information flows and evolving information tactics. Efforts will culminate in an LLM/AI-powered software with the ability to collect varied and real-world information types across multiple vectors of media in a dynamic environment that can assess and generate a psychographic model of a population based on contextualized sentiment analysis. It will present a user interface that is either traditional or virtual, that permits a customer to provide inquires and prompts to direct the software towards focused analysis. Lastly, it will demonstrate the capability for users to visualize maneuvers, make recommendations on what language might alter current sentiments, and/or forecast impacts of potential courses of action and events on target audiences. REFERENCES: 1. Ng, Lynnette H.X. & Carley, Kathleen M. (2022). Is my stance the same as your stance? A cross validation study of stance detection datasets. Information Processing & Management, 59, 6. [DOI] WebSite: [link] 2. Ng, Lynnette H.X. & Cruickshank, Iain J. & Carley, Kathleen M. (2022). Cross-platform information spread during the January 6th capitol riots. Social Network Analysis Mining, 12(1), 133. [DOI] WebSite: [link] 3. Williams, Evan M. & Carley, Kathleen M. (2022). TSPA: Efficient Target-Stance Detection on Twitter. Proceedings of the 2022 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 242-246. [DOI] WebSite: [link] 4. Krugman, Jan O. & Hartman, Jochen (2024). Sentiment Analysis in the Age of Generative AI. Customer Needs and Solutions, 11, 3. WebSite: [link 1] OR [link 2] 5. Plaue, Matthias (2023). Large-scale language models for innovation and technology intelligence: sentiment analysis on news articles. Mapegy. WebSite: [link] 6. Mets, Mark & Karjus, Andres & Ibrus, Indrek & Schich, Maximilian (2024). Automated stance detection in complex topics and small languages: The challenging case of immigration in polarizing news media. PLoS ONE, 19(4). [DOI] WebSite: [link] 7. Juros, Jana & Majer, Laura & Snajder, Jan (2024). LLMs for Targeted Sentiment in News Headlines: Exploring Different Levels of Prompt Prescriptiveness. Arxiv. WebSite: [link] 8. Lynch, Christopher J. & Jensen, Erik J. & Zamponi, Virginia & O'Brien, Kevin & Frydenlund, Erika & Gore, Ross (2023). A Structured Narrative Prompt for Prompting Narratives from Large Language Models: Sentiment Assessment of ChatGPT-Generated Narratives and Real Tweets. Future Internet, 15(12), 375. [DOI] WebSite: [link] 9. Namikoshi, Keiichi & Filipowicz, Alex & Shamma, David A. & Iliev, Rumen & Hogan, Candice L. & Arechiga, Nikos (2024). Using LLMs to Model the Beliefs and Preferences of Targeted Populations. Arxiv. WebSite: [link] 10. Zeng, Jingying & Huang, Richard & Malik, Waleed & Yin, Langxuan & Babic, Bojan & Shacham, Danny & Yan, Xiao & Yang, Jaewon & He, Qi (2024). Large Language Models for Social Networks: Applications, Challenges, and Solutions. Arxiv. WebSite: [link] 11. Zhang, Peter (2023). Networking and LLM in the Age of AI. Medium. WebSite (not functional on government computers): [link 1] AND [link 2] 12. Tang, Yunlong & Bi, Jing & Xu, Siting & Song, Luchuan & Liang, Susan & Wang, Teng & Zhang, Daoan & An, Jie & Lin, Jingyang & Zhu, Rongyi & Vosoughi, Ali & Huang, Chao & Zhang, Zeliang & Zheng, Feng & Zhang, Jianguo & Luo, Ping & Luo, Jiebo & Xu, Chenliang (2024). Video Understanding with Large Language Models: A Survey. Arxiv. WebSite: [link] 13. Blane, Janice T. (2023). Social-Cyber Maneuvers for Analyzing Online Influence Operations. Carnegie Mellon University. WebSite: [link] 14. Jiang, Julie & Ferrara, Emilio (2023). Social-LLM: Modeling User Behaviors at Scale using Language Models and Social Network Data. Arxiv. WebSite: [link] KEYWORDS: social media; analytics; news sources; visualization; modeling; large language models; forecasting; classification; assessment; encoding; influence; characterization; communities; networks; information maneuvers; influence campaigns; information operations; cognitive warfare; artificial intelligence; wargaming; immersive simulation; great power competition; gray zone; sentiment; recent events; psychological operations; social psychology; sociology; culture; prediction; emotions; opinions; geography
