CASE STUDY

Saving lives with a virtual reality and augmented reality counter-IED classroom

Let's be honest for a moment.

For most of us, the thought of handling explosives built by someone who's primary goal is to cause maximum damage to you and those around you is a less-than-fun idea.

But for some brave people, the constant threat of improvised explosive devices (IEDs) is a waking reality.

IEDs have become one of the biggest threats to servicemen and civilians in recent war zones, accounting for ~63% of fatalities to coalition forces in Iraq, and 66% in Afghanistan. The key feature of IEDs is they’re difficulty to identify, and can be disguised as anything from bricks in a wall to buried under roads.

My responsibilities

As the lead UX/UI designer, it was my responsibility to:

• Gather user insights
• Identify user needs
• Propose flows that onboard new users and retain existing users
• Propose interfaces as low-fidelity sketches and wireframes
• Define the visual style and interaction methods
• Create high-fidelity interface mockups and prototypes

tl;dr

Cutting to the chase: we designed and built 3 experiences

Instructor console

Every classroom, even a virtual one, has an instructor.

But what if that instructor could see where students are struggling in real-time? When not getting hurt out in the real world is the fundamental goal of your lesson, you need to have confidence that your students are ready to step outside.

Skip to more about the Instructor Console

The challenge

How might we use technology to create a safe and immersive space where people can learn how to identify and avoid IED threats?

Insights

• Military personnel are especially competitive
• Tech literacy is high within the military
• Scalability is crucial, as new forms of IED are discovered regularly
• The solution must be flexible to accommodate both classroom and field environments

Solution 1: Augmented reality petting zoo

The most important part of hands-on learning with dangerous things is to remove the danger.

By combining a mobile device with a camera such as an iPad, and a game engine such as Unity3D, you're able to create a handheld virtual window into an enhanced, mixed-reality version of the real world. Bringing virtual objects into the real world gives us a better sense of scale and a far more immersive experience than a picture or video on a screen.

Inside and outside of the classroom

The augmented reality petting zoo is intended for use both in the classroom and in the field so that lessons can be held whenever time allows. For this reason, deploying the virtual classroom must be fast, and the duration of lessons flexible to accommodate a dynamic schedule.

Solution 2: Virtual reality game

Virtual reality (VR) can be summarised as: "Hijacking the senses to transport you into a world entirely detached from your actual location.” In the Counter-IED VR game do this by hijacking sight, sound, touch, sense of scale, balance and acceleration.

(Reach out to me if you’re interested in hearing more - I have a whole conference talk selling the future of VR, if you fancy being transported back to 2015 when VR was this world changing concept.)

So you've learnt how to identify IEDs in the Petting Zoo. Now it's time to put that to the test in the 360º Virtual Testing Environment.

The goal is to make personnel feel totally immersed in a hostile environment so that the identification skills they learnt in the Petting Zoo may be tested in a real-but-safe simulation. Throughout user testing, one test user - a member of the US armed forces - experienced a panic attack brought on by the immersive environment. Whilst I don't particularly want to upset a person with deployment and firearm experience, you could say our level of immersion reached a high benchmark.

In 2014, VR kits were extremely expensive, big, and cumbersome. Whilst the experience we built for the Oculus Rift was exceptionally immersive, a cheaper and more mobile version that scales easier was needed. Fortunately (both for the USA military, and my ability to screenshot apps) the iPad is once again suited to the task. The screen and accelerometer could once again combine to offer a virtual window. But instead of enhancing the real world as in the AR Petting Zoo, it could be used as a porthole into another world.

Solution 3: Instructor console

When you're responsible for the safety of your colleagues and students, you want the confidence that they'll be safe outside of the classroom. The instructor console provides historic and real-time performance information so that knowledge gaps can be identified long before they become fatal.

The iPad app compliments both the Petting Zoo and VR Virtual Testing Environment by allowing instructors to review quantitative metrics related to entire classes or individual students, view their activity in real-time, and intervene where necessary.

Project credits

Studio: Hidden Ltd
Concept art: Daniel Burns, Patrick Robert Doyle
3D modeling: Daniel Burns, Martin Ashford