FRIDAY: Unlocking the Power of Open-Source Intelligence for a Data-Driven Army

Introduction

The modern intelligence landscape is characterized by unprecedented opportunities and challenges. The sheer volume of publicly available information (PAI) and open-source intelligence (OSINT) offers invaluable insights into emerging threats and complex operational environments where collection assets are limited. Effectively harnessing PAI and OSINT at speed and scale requires overcoming significant hurdles, particularly when integrating unclassified information into classified intelligence workflows. To take full advantage of OSINT and enable near-real-time intelligence analysis, the U.S. Army must prioritize both technical interoperability and policy reform to streamline the flow of open-source data into classified analysis.

The FRIDAY project, developed by Southern European Task Force—Africa’s (SETAF-AF) Africa Data Science Center (ADSC), enables the seamless and secure movement of OSINT data from the unclassified Non-Secure Internet Protocol Router Network (NIPRNET) to the classified Secret Internet Protocol Router Network (SIPRNET), where analysts can use the data to enhance object-based intelligence production within existing enterprise systems like the Army Intelligence Data Platform (AIDP). Produced through a collaboration between military personnel and civilian data scientists, FRIDAY utilizes a novel data processing capability to overcome interoperability limitations in current intelligence programs of record, enabling a holistic and data-driven approach to intelligence analysis. FRIDAY addresses the critical need for rapid and timely conversion of open-source data into actionable intelligence on classified systems. SETAF-AF is thus empowered to capitalize fully on the wealth of information available in the open-source environment, which ultimately strengthens the overall security posture within its area of responsibility.

Currently, turning raw OSINT data into actionable intelligence objects within AIDP involves a series of multiple handoffs between different teams and systems. This fragmented approach risks creating bottlenecks, increases the potential for errors, and limits the speed and agility of the intelligence cycle. FRIDAY tackles the fragmented multi-domain challenge head-on through a streamlined process using existing government off-the-shelf (GOTS) and commercial off-the-shelf (COTS) systems that are readily accessible to the Army intelligence community. Instead of relying on cumbersome and error-prone manual creation and re-creation, FRIDAY automatically and securely moves OSINT reports from NIPRNET to SIPRNET, eliminating a significant workflow bottleneck for analysts. This not only frees up bandwidth for analysts but also ensures that analysts operating within classified environments have ready access to valuable OSINT insights. Additionally, the FRIDAY project addresses the critical need for secure information sharing between different classification domains using existing enterprise tools with defense and intelligence community standard authentication methods and group- and role-based access controls.

As shown in the figure above, FRIDAY is not just a tool but a pipeline that features a user interface for data entry, an environment for data processing, a cross-domain solution, and, at the final step, integration with AIDP. Recognizing that OSINT input often comes in inconsistent formats, FRIDAY implements a crucial step: data normalization. As the OSINT collector enters OSINT reports, FRIDAY standardizes their format. Ensuring data consistency and ontology compatibility regardless of the original source or structure makes the data resistant to “anomalous usage patterns found in intel traffic.”¹ Data normalization is a requirement for seamless integration with enterprise tools like AIDP to allow OSINT data to corroborate other sources for intelligence purposes.

Once FRIDAY processes and transfers OSINT data to the SIPRNET environment, it undergoes a crucial transformation into “ontology objects” within AIDP. These objects represent key entities, events, and units extracted from the OSINT reports, and they enrich the existing intelligence picture with valuable insights gleaned from the open-source realm. This object-based approach goes beyond simply adding more data; it connects the dots between seemingly disparate pieces of information. By linking OSINT-derived objects with classified data already present in AIDP, analysts achieve a more comprehensive understanding of the operational environment.

During design and development of the pipeline, ADSC conducted tradeoff analyses and testing with multiple technologies to prototype a solution. Advana, Chat Surfer, and Microsoft Power Platform are already acquired GOTS and COTS systems, allowing ADSC to bring the tool from design to user-acceptance testing in under four months with zero added cost to the organization. While the current implementation (outlined in solid black in the figure referenced by the status indicator) balances robustness and feasibility given the availability of the tools, further iteration and analysis are required to attain long-term viability and scalability.

Ultimately, the goal is to establish near real-time connectivity between open-source information and classified analysis. To unlock FRIDAY’s full potential, the U.S. Army must break down the barriers to true interoperability, which present in two categories: technical and procedural. Technical interoperability requires compatible schema definitions, practical ontologies, data governance and security best practices, and avoiding vendor lock-in. Procedural interoperability is sometimes more difficult to achieve. It requires different organizations with idiosyncratic, people-driven processes to design systems using common or compatible technical specifications and to embrace VAULTIS data standards,² often entailing daunting cultural shifts on top of technical project setup. Further, organizations must designate stewards to take responsibility for data initiatives beyond initial operating capability and into the maintenance phase.

Compatibility issues in integrating data science and engineering (DS&E) tools with, for example, legacy systems present data formatting discrepancies and security challenges that hinder the smooth exchange of information between new and existing systems. Addressing these interoperability hurdles requires a strategic approach that identifies technical and procedural limitations and deliberately weighs the costs of long-term solutions against the risks and opportunities of short-term workarounds.

While the benefits of integrating DS&E into intelligence workflows are undeniable, we must continue to highlight additional challenges to fully realize its transformative potential. One hurdle is overcoming cultural resistance to new technologies and approaches. Many intelligence professionals steeped in traditional methods may be hesitant to embrace DS&E, perceiving it as disruptive or overly complex. Therefore, fostering a deeper understanding of DS&E capabilities among both analysts and leadership is crucial.³ This requires demonstrating the tangible value of DS&E through concrete examples and success stories, highlighting its ability to enhance, not replace, existing expertise.

Conclusion

Finally, building a sustainable pipeline of skilled data science professionals is paramount for long-term success. This requires a multifaceted approach that encompasses targeted training programs for existing intelligence personnel, recruitment efforts aimed at attracting top data science talent, and the establishment of career paths that recognize and reward expertise in both intelligence and DS&E. By investing in workforce development, the intelligence community can ensure it has the skilled personnel necessary to leverage the power of data science effectively for years to come.

Initiatives like FRIDAY demonstrate the transformative potential of DS&E in modernizing intelligence operations, particularly in leveraging the power of OSINT. By examining the factors that inhibit innovation within the enterprise, and encouraging data-driven solutions, the U.S. Army can maintain its strategic advantage in the face of evolving threats and complex operational environments. Continued investment in DS&E infrastructure, training, and research will be critical for ensuring timely, insightful, and actionable intelligence reaches decision-makers at all levels.

Endnotes

1. J. Palmer, “Textually retrieved event analysis toolset,” MILCOM 2005—2005 IEEE Military Communications Conference Vol. 3, Atlantic City, NJ, 2005, 1679-1685. https://ieeexplore.ieee.org/document/1605916.

2. “VAULTIS” is an acronym for “visible, accessible, understandable, linked, trustworthy, interoperable, and secure.” For more information on VAULTIS standards, see Rebecca Sammons, “Laying the Foundation for AI Adoption in the Department of Defense with the VAULTIS Framework,” Government Technology Insider, 23 April 2024, https://governmenttechnologyinsider.com/laying-the-foundation-for-ai-adoption-in-the-department-of-defense-with-the-vaultis-framework/.

3. Chris Tomlinson, Felix Rodriguez Faica, Ryan Harvey, and Keith Hickman, “Modernizing Intelligence Operations in Africa: Enhancing the Intelligence Cycle through Data Science,” Military Intelligence Professional Bulletin PB 34-25-1 (January-June 2025), 39-44, https://mipb.ikn.army.mil/issues/jan-jun-2025/modernizing-intelligence-operations-in-africa/.

COL Christopher Tomlinson currently serves as the Director of Intelligence, G-2 for the Southern European Task Force, Africa and is operational director of the Africa Data Science Center for SETAF-AF. His prior intelligence assignments include Director of Intelligence, J-2 Special Operations Joint Task Force—Operation Inherent Resolve, Deputy Director of Intelligence Joint Staff J-2, and Theater ACE Chief USAREUR. He completed a master’s in strategic studies from the Marine Corps War College and a BA in political science at Texas Tech University.

CW3 Felix Rodriguez Faica currently serves in the Intelligence Operations Division of the Southern European Task Force, Africa G-2 as an Intelligence Planner and Common Intelligence Picture/Army Intelligence Data Platform lead integrator. His previous assignments were at various unit echelons to include brigade combat team and MI brigade-theater. He received a BA in intelligence studies at American Military University and completed the Digital Intelligence Systems Master Gunner Course.

Angela White is a Data Scientist for the ADSC of the Southern European Task Force, Africa G-2. She previously worked with the Joint Staff Office of the Chief Data Officer to integrate and implement data systems for multiple directorates of the Joint Staff. Before becoming a Data Scientist, Angela studied astrophysics at the University of Pennsylvania and served as an Electronic Warfare Systems Engineer and physicist.

Kathryn Ruhl is a Data Engineer for the Africa Data Science Center. Prior to the ADSC, she worked as a data scientist at the National Geospatial Intelligence Agency and a DevOps engineer deploying various enterprise cloud applications within the IC. Kathryn studied economics at George Mason University and is also a U.S. Marine.