Virtual Reality System Trains Aircrews to React to Threats

Threat reaction training of aircrews is a critical factor in protecting aircraft and crews from ground-based missiles and other weapons. To help provide this training, GTRI researchers are working with the 189th Airlift Wing (AW) of the Arkansas National Guard to develop a high-fidelity immersive multi-player simulation of the battle airspace.

The 189th Airlift Wing is an active duty organization aligned with Air Mobility Command, and is the center of legacy training in the Air National Guard. Based in Little Rock, Ark., its mission is to provide premier training to and certification of the C-130 community, consisting of pilots, navigators, flight engineers, and loadmasters.

GTRI has been performing sponsored research for the 189th AW Innovation Cell led by Lt. Col. Justin Fitzpatrick as the Wing’s Innovation Lead. The goal of this research has been to design and develop a system that allows aircrews to repeatedly practice threat responses with a level of realism that simulates as closely as possible what happens in real-world missions. The high level of realism helps convert correct aircrew responses into reflexes bordering on rote learning and memorization.

VIDEO: FITRE project overview and demonstration.

Developing the Fully Immersive Threat Reaction Environment

The research has led to the development of the Fully Immersive Threat Reaction Environment (FITRE) system with a goal of revolutionizing the way that the crews train. FITRE, at the basic level, is a multi-player video game consisting of four players that takes place in virtual reality. An instructor orchestrates the C-130H training mission and controls the parameters of the game. A pilot is placed into a high-fidelity C-130H virtual cockpit that includes a missile warning system (MWS) simulation and all necessary flight instruments.

A loadmaster is placed on either side of the back of the C-130H and operates a realistic flare dispensing switch. And finally, a shooter is placed on the ground of the mission space and is able to operate a number of weapons, which include shoulder-fired surface-to-air missiles and large-caliber anti-aircraft machine guns. The shooter has the ability to easily move around the mission space to obtain the desired vantage point.

Front view of male researcher wearing VR goggles with hands on flight controls, in a darkened room

A GTRI researcher demonstrates a virtual reality simulation of the battle airspace built to help train C-130H air crews on responding to threats. (Credit: Sean McNeil, GTRI)

A high-fidelity simulation framework provides authoritative threat simulation capabilities that enable the modeling and simulation of the supported weapon systems within FITRE and their kill chains at the engagement and mission levels. FITRE supports both day and night modes. Advantages during night engagements can be realized by the trainees when operating from the shooter's perspective, while nighttime aircraft flare dispenses noticeably increase exposure of the aircraft to the shooter.

In the virtual cockpit, necessary aircraft details improve navigation effectiveness of the pilot while visual and audio declarations identify encountered threats during engagements. All these details help the crew engage with the simulation.

“Because this type of training is all about the quick reaction to specific stimuli, if the stimuli during training don’t resemble the real thing, the crew may feel like they are experiencing this very dangerous situation for the first time. The key is to simulate the cognitive load as accurately as possible,” said Izudin Ibrahimbegovic, a GTRI senior research scientist who led the development of the system.

Male researcher standing in front of two large video monitors, wearing VR goggles and holding a control device in front of him

A simulated shooter demonstrates the battle airspace from the ground, with a simulated shoulder-fired missile and aircraft on displays showing what the VR headset is presenting. (Credit: Sean McNeil, GTRI)

Evaluating Air Crew Response to the Threats

Because so much happens in very limited time, instructors can have difficulty evaluating how the aircrew responded to the threats. The FITRE system helps with evaluating the participants’ performance by tracking eye movements, head movements, and flight control inputs.

“Not only can we can tell them what they should have done during the first seconds of the engagement,” said Andrew Braun, the lead systems engineer on the project, “but now we can show them exactly how it should be done. We can also focus on assisting in the evaluation of the crew as a whole during these situations.”

Threat reaction training is currently done with a much lower-fidelity system that does not include an immersive virtual reality cockpit or alternate views of the training space. Threat reaction training can also take place during flights into test ranges. However, the number of range opportunities is limited and requires significant investments of time and money – and comes with inherent risks.

Male researcher sitting at table containing FITRE hardware, there is a large video screen, and he is wearing VR goggles and holding a control stick

A simulated pilot’s view of a cockpit seen through virtual reality headset.

Expansion, Future Enhancements Planned

So far, FITRE has been developed only for the C-130H aircraft, which is what the 189th Airlift Wing operates and trains for. However, the system can be adapted for other fixed-wing and rotary-wing platforms to enable crews of those aircraft access to threat reaction training.

GTRI researchers developed the simulation using publicly available terrain data. If the project receives additional funding, plans exist to expand scenery to simulate other geographic locations around the world.

Beyond expanding the number of aircraft and geographic backgrounds, the researchers are also working on connecting systems to allow multiple crews to train together for multi-ship missions.

Hands of researcher on aircraft controls in lab setting.

Simulated aircraft controls allow pilots to fly an aircraft in virtual reality battle airspace for an engagement between a shooter on the ground and a C-130H air crew. (Credit: Sean McNeil, GTRI)

The simulation runs on commercial-off-the-shelf hardware, including standard headsets and laptop computers. FITRE training can be done in very confined spaces and the system setup can be accomplished in under an hour by a trained team.

Ibrahimbegovic and his team have been demonstrating FITRE to interested groups and receive a positive reception every time.

Project Resulted from Partnerships

The research project also has been supported by the Advanced Airlift Tactics Training Center (AATTC), Air National Guard Air Force Reserve Command Test Center (AATC), ARCWERX (the innovation cell of the Air Force Reserve Command), and the Defense Intelligence Agency.

Group of 4 male researchers standing in a row posed for photo, background is FITRE setup with table and two large video monitors on wall

A GTRI research team has worked with the 189th Airlift Wing to develop a high-fidelity immersive multi-player simulation of the battle airspace. Shown (l-r) are: Matthew Hansen, Izudin Ibrahimbegovic, Andrew Braun, and Robert Plante. (Credit: Sean McNeil, GTRI)

Writer: John Toon (john.toon@gtri.gatech.edu)
GTRI Communications
Georgia Tech Research Institute
Atlanta, Georgia USA