Big Data Enables Artificial Intelligence on Army Networks

Indiana National Guard soldiers with the 38th Infantry Division, Maj. Adam Barlow, operations chief from Shelbyville, and Capt. Justin Shutt, battle captain from Fort Wayne, monitor a simulated battle during the division’s command post exercise at Camp Atterbury near Edinburgh, Indiana, in February. The Unified Network is foundational to Army modernization efforts. Credit: Master Sgt. Jeff Lowry

By reducing the number of networks and expanding big data capabilities, the U.S. Army is enhancing cybersecurity and laying the groundwork for using artificial intelligence (AI) on the network.

In recent years, the service has doubled the amount of network endpoints contributing log data nearly in real time into the big data platform known as Gabriel Nimbus, which is based on the Defense Information Systems Agency’s big data platform, an open-source system that supports the data ingest, correlation and visualization infrastructure. The big data platform common architecture can be installed across hundreds of servers in several hours and enables data, visualizations and cyber analytics to be shared with mission partners.

The Army also continues to increase the number of data feeds and was scheduled this summer to discuss the next tranche of data feeds to be integrated “to really enrich what we have available and really improve what our analysts will be able to do,” reported Lt. Gen. Maria Barrett, commander, U.S. Army Cyber Command, in a SIGNAL Media interview.

She added that the service has also doubled the amount of storage and increased the number of analysts using the big data platform. “I see this as really pivotal to what we do not only in the defense of our network but in a couple of other areas as well. As we continue to mature the data that is in this data platform, we are really then going to be able to posture us to insert automation and, eventually, artificial intelligence into our network. You can’t get there without having the requisite data available.”

Gen. Barrett and other command leaders tout AI’s ability to continuously monitor networks and systems. “I think AI has potential for how we do continuous monitoring, especially as we move more and more of the Army’s applications and services into the cloud. “If we know that a common tactic, technique and procedure of the adversary is to do actions one, two and three, we can now take a look at the data that would support that and can start to automate that and potentially leverage AI for that as well. I think there’s huge potential there.”

The commander said she sees the effort as part of a multiyear plan but added, laughing, that she doesn’t want to be held to a specific timeline.

The command already is in the early stages of building an AI system for continuous cyber monitoring, revealed Mark A. “Al” Mollenkopf, Army Cyber Command’s science advisor to the commanding general, in a separate interview. “We’re thinking that we’re at the point now from an artificial intelligence perspective, where continuous monitoring can be plugged into existing data platforms and seems to enhance the visibility and security writ large.” (See AI May Benefit Cyber Defense More Than Offense).

Command officials are working with the office of the Assistant Secretary of the Army for Acquisition, Logistics and Technology, ASA (ALT), on security and continuous monitoring of weapon systems. “ASA(ALT), I think with our advice, will figure out how to secure the platform and all weapon systems as appropriate,” said Steven Rehn, the Army Cyber Command chief technology officer.

Rehn explained that a continuous monitoring system will provide alerts so that commanders can see what is happening on their networks and “make a risk determination on how to execute and operate.”

Gen. Barrett used a metaphor to illustrate the network visibility provided with big data: “In the past, we have been asked to respond to certain Army networks, and those networks may have been operated separately from the rest of the enterprise. And as such, they might not have had the right sensors in place. They might not have had the requisite amount of data for us to look at. It’s like walking into an empty room. There’s really nothing for us to describe about the room.”

In such cases, officials must generate the requisite data after the cyber incident. “It’s after the fact, and it’s not a great way of operating,” Gen. Barrett declared.

Rehn linked the extended Gabriel Nimbus to the Army’s efforts to build a tactical data fabric, which weaves together numerous information sources and data formats from different systems, providing a common layer to improve interoperability and quickly route the right data to the right operator. A data fabric should reduce digital barriers between warfighting functional systems such as fires, maneuver, air defense, or sustainment and enrich the pool of data available for AI and machine learning to aid decision-making.

“We’ve been able to pick up technology and push it into edge computing devices. We call that the lower echelon tactical analytic platform,” Rehn said, adding that it had been tested during Project Convergence exercises in 2021 and 2022. “The result of that was basically the foundation right now of where the Army is moving towards a tactical data fabric.”

The data fabric will move the Army away from a message-centric approach to data sharing in which a message is sent to a particular person or group. Instead, information added to the data fabric will be readily available to all who have access. “This makes the data available to a point in the data fabric so that now you can access that data and begin processing on the data. There may be data that’s now available to you as pertinent for your mission area, which in previous times was not available,” Rehn elaborated.

Gen. Barrett also lauded efforts over the last three years by Lt. Gen. John Morrison, Army deputy chief of staff/G6, and Chief Data Officer David Markowitz, who also serves as the Army’s acting chief information officer, to collapse the Army’s vast array of enterprise and tactical networks into one Unified Network. “There is this huge effort over the last two or three years to collapse these smaller networks, or individual networks, that have been operated by organizations and collapse them into the enterprise, therefore enabling Army Cyber to see those networks and ensure that they are censored and instrumented appropriately, and we’re ingesting that data into the big data platform.”

Collapsing networks from various Army organizations provides a number of benefits, including saving money and enhancing cybersecurity, she indicated. “We’re getting the feeds from that. We’re looking at their boundary security. We’re looking at their forward-facing websites. And so that then opens up the possibility for me to monitor them consistently, and really at a level that is usually better than what they were doing before,” the commander noted. “It lets John Morrison use more enterprise license agreements, which give me the tools that we need, etcetera, etcetera, etcetera.”

The enhanced visibility gained by consolidating disparate networks gives her greater confidence the service can identify cyber threats. “That ability to look at data that we had from the network to either find adversary activity or confirm with a high level of confidence that an adversary is not in our network—which is equally if not more valuable—having that data available to the analysts is really vital,” she said.

One recent step toward network consolidation came in the form of a signed memorandum of understanding that allows the Army Cyber Command and the Enterprise Cloud Management Agency (ECMA) to work together more closely. Army officials consider the deeper integration between the organizations as a positive step toward achieving key convergence objectives in support of the Army’s Unified Network Plan. The primary outcome of the agreement enables ARCYBER to direct routine or urgent defensive actions for Army cloud, or cArmy, consistent with the rest of Army networks. It also provides ECMA greater direct access to other resources, such as critical cyber intelligence.

The sheer number and variety of networks are rooted in the Army’s first use of the Defense Department’s non-secure and secret networks known as NIPRNet and SIPRNet. “Some of the products that we were using didn’t scale, and so there were enclaves that were stood up, and then they were connected to each other. But they were different,” Gen. Barrett said.