GTRI Uses AR and Line-of-Sight Calculations to Outsmart the Enemy

A photo of GTRI researcher Emily Strube.

As adversaries get stealthier on the battlefield, the need for warfighters to remain vigilant against potential attacks and to indicate safe routes for troop movements that minimize exposure to and observation by the enemy are crucial. This would improve response times, operational tempo, and survivability of operational forces.

To address this challenge, the Georgia Tech Research Institute (GTRI) is working to increase the situational awareness of troops on the ground through the use of augmented reality (AR) devices and line-of-sight calculations within environments through the use of Unity, a cross-platform game engine developed by Unity Technologies. The project would facilitate improved nonverbal and distance communication between troops about the location of adversaries during operations.

The work has been supported by GTRI’s Independent Research and Development (IRAD) program and won an IRAD of the Year award in fiscal year 2022.

"Let's say you're a part of a squad that's stationed on a rooftop and another squad in your company is stationed elsewhere," said Emily Strube, a GTRI research scientist who is leading the project. "As the other group radios in to your team, bombs start dropping on their location and you can't hear them over the radio and they can't hear you. You observe a group of enemies coming their way, but you don't want to warn in a way that could alert the adversaries. This is the type of situation our project seeks to address."

The work calculates the line of sight of enemies by using a computer application to draw lines from an enemy's position(s), connects those lines on a polygon mesh, and then colors the meshes based on the enemy's visibility.

The visibility representation ranges from black to gray to white. Areas that are black are visible to no enemies, areas that are white are visible to all enemies, and areas that are gray are visible to some number of enemies.

The colors are then displayed using Unity both within a computer application and a HoloLens-based application. HoloLens is a pair of mixed-reality smartglasses developed and manufactured by Microsoft.

"The Microsoft HoloLens is basically just a pair of glasses that project light in front of your eyes so that you can see holograms in the real world," Strube said.

The project utilizes an existing GTRI service called Realtime Intelligence Fusion Service (RIFS), which was built with Unity and has HoloLens functionality. RIFS produces spatial information about a room and then displays that information as meshes that are visible to a command center on a desktop computer.

From there, the project uses a pathfinding algorithm to generate a path for each HoloLens user to a goal point that is least visible to the enemy. Then, a common operating picture of the area is shared among all HoloLens users – in this case, the warfighters – as well as a command center. The common operating picture consists of the meshes, the visibility metric displayed on them, the generated paths, and the tagged enemy positions.

"Going back to my initial scenario, the troop that's on the roof can now point out the enemy and tag them so that the troops on the ground can know what's there before it gets to them," Strube said.

"Additionally, lines of sight can be calculated for those enemies and a new path can be generated for those troops on the ground to go from their current position to their goal position in a way that is least visible to the enemy."

The battlefield is just one of many use cases to which this research can be applied.

GTRI is also looking to incorporate the work into warehouses, specifically smart warehouses. Similar to smart homes, smart warehouses are enabled with various automated and interconnected technologies. GTRI's pathfinding algorithm could help workers more easily pick up and drop off packages at drop points within a warehouse and also help people locate components inside warehouses to complete repair orders.

The project could also help users locate and maneuver around chemical hazards.

The research is currently being incorporated into various programs.

Writer: Anna Akins (anna.akins@gtri.gatech.edu)
Photo Credit: University of North Georgia
GTRI Communications
Georgia Tech Research Institute
Atlanta, Georgia USA