OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Human-Machine Interfaces;Trusted AI and Autonomy OBJECTIVE: Utilize state-of-the-art artificial intelligence (AI) technologies (e.g., Large Language Models) to develop an AI-aided human-in-the-loop instructional systems design agent/coach, support conversion of legacy (e.g., PowerPoint and Word) course materials, and assist in generation of content within Marine Corps learning ecosystem (i.e., Moodle) to update/revise or develop new courses. DESCRIPTION: In January 2023, the Marine Corps released Training and Education 2030 that begins with the statement The current [Training & Education] T&E system is not preparing the Marine Corps for the future operating environment. [Ref 1]. Today's training is built on an industrial-era model where students move through a one-size-fits-all production line, progressing through standardized programs of instruction (POIs) comprised of static slides, student handout documents, written exams, and minimal experiential learning. The same report then goes on to state that better technology integration in our classrooms and courseware can increase production, produce a more highly trained Marine, reduce their overall time-to-train, and limit the burden on our learning infrastructure. The recent advancement of AI technologies including Large Language Models (LLMs) and image generators such as Stable Diffusion provide an opportunity to aid Marine Corps transition from industrial to information age learning in three ways: (1) instructional systems design assistance (e.g., develop course outlines, map Knowledge, Skills, and Abilities [KSAs] to tests and quizzes, advise personnel on how to best build instructional exercises in their eLearning environment, etc.) to aid course designers, instructors, and others in revising and bootstrapping course outlines and course content to reduce challenges for personnel (e.g., intimidated by the blank slate of creating content from scratch); (2) content generation assistance for creation of multimedia and/or interactive learning aids that includes animations and videos, but also goes beyond the creation of quizzes and flashcards that are possible with today's LLM technologies to generate branching scenarios, and other more interactive content; and (3) conversion to bring legacy content (e.g., text, slides, images, infographics, complex diagrams, etc.) into the modern learning environment. The overarching goal of this SBIR topic is not to use AI as a standalone replacement for curriculum developers, POI managers, or instructors, but to enable human-AI teams to develop and manage training for the Marine Corps faster and more effectively than current processes. Technology created from this effort is expected to show efficiency gains in content creation and course generation without negatively impacting learning outcomes. Proposed solutions must go beyond the existing commercial and open-source efforts to integrate AI into learning platforms [see Ref 2 as an example]. Proposed solutions must create desired modern, interactive-content AI models that can accept input from a variety of sources and formats and output not just text, but images, movies, and more that must work together seamlessly. The end state of this effort is to provide a government-owned suite of AI-enabled software capabilities for use by the Marine Corps Training and Education enterprise to more efficiently and effectively create courses and content as part of modern e-learning systems that reflects a cutting-edge, information-age learning enterprise. A good example are Marine Corps maintenance schoolhouses whose courses currently include a long classroom component (e.g., PowerPoint slides and lecture) with some hands-on practical application time (e.g., performing the maintenance task on a physical system). The desire is for the classroom components to be richer with multimedia and interactive components (e.g., branching scenarios allowing students to interactively troubleshoot issues); however, creating these from scratch or converting static text or images is time consuming. Additionally, even converting a Program of Instruction into a new, blank Moodle course is daunting. The end goal is a tool that helps personnel take their current POI, slides, and documents and creates a new course and populates it with content. Personnel will always be in the loop to verify, modify, and add to AI-created content, but AI technologies that can bootstrap the course and content creation and conversion process would be a significant savings of time and effort. Due to the potential long review times involved, human subject research is discouraged during Phase I. Phase II plans should include key component technological milestones and plans for at least one operational test and evaluation, to include user testing. PHASE I: Develop early concepts, wireframes, workflows, and requirements for AI-enabled modern training and education software capabilities to include support for instructional systems design processes (e.g., creating course outlines in eLearning systems based off a Marine Corps POI), content (i.e., multimedia and interactive) creation, and legacy (i.e., static PowerPoint and Word) content conversion. Show how the human-AI team will work together to produce training content, since the human designer is an essential part of the development process. Human factor and human subject testing are critical in follow-on Phases of this topic. Please carefully review the requirements of approval for proposals that include testing of human subject and compliance with Institutional Review Board (IRB) [Refs 3, 4]. PHASE II: Conduct an evaluation with Marines (coordination aided by ONR) of AI-enabled capabilities outlined during the Phase I evaluation to include a usability assessment, process improvement demonstration, and effectiveness evaluations where appropriate. Perform additional demonstrations/experimentation that show system extensibility through plug-and-play of new and updated AI models to improve and expand upon overall system capability. Collect impressions of usability and develop objective metrics of time and effort to create and convert courses and content with a relevant Marine Corps population (i.e., including Marine Corps government and civilians in-the-loop of the course and content creation process). Perform all appropriate engineering tests and reviews, including a critical design review to finalize the system design. Produce the following deliverables: (1) a working prototype of the system that can interact with existing system specifications; (2) evaluation of system usability and efficiency to convert and create content and courses; (3) a system effectiveness evaluation of system capabilities to provide demonstrable improvements (Human Subjects protocol needs to be approved in Phase II Option if needed for this evaluation). PHASE III DUAL USE APPLICATIONS: Support the Marine Corps in transitioning the technology for Marine Corps use. Develop the software for evaluation to determine its effectiveness in either a formal Marine Corps schoolhouse or other training setting. As appropriate, focus on broadening capabilities and commercialization plans. Development of affordable, scalable, non-proprietary technologies are needed to accelerate the transition of the Marine Corps to an information age training model. The commercial sector is developing some of these AI-enabled T&E technologies, but they often do not deal with critical issues regarding non-existent, limited, or low-quality source data, do not address encryption and classification requirements, and often come with prohibitive licensing and usage fees. This technology will have broad application in the commercial sector. Specific examples of businesses in the commercial sector include Moodle (which has a commercial learning services component) and Docebo (an online learning platform for enterprises). REFERENCES: 1. Training and Education 2030. Department of the Navy/United States Marine Corps, January 2023. https://www.marines.mil/Portals/1/Publications/Training%20and%20Education%202030.pdf?ver=G6MJLpoB3_H4JRuo0FwthA%3d%3d 2. Porritt, Matt. Announcing Moodle LMS 4.5: Unlocking the power of AI. Moodle Community News, October 7, 2024. https://moodle.org/mod/forum/discuss.php?d=462651 3. Office of Naval Research. Human Subject Research Documentation Requirements . https://www.onr.navy.mil/work-with-us/how-to-apply/compliance-and-protections/research-protections/human-subject-research 4. Office of Naval Research. Research Protections . https://www.nre.navy.mil/work-with-us/how-to-apply/compliance-and-protections/research-protections KEYWORDS: Artificial intelligence; machine learning; AI/ML; content creation; training and education; instructional systems design; large language models
