During his popular 2015 TED Talk, immersive artist, entrepreneur, and director Chris Milk suggested that virtual reality could potentially be the “ultimate empathy machine.” This is something Milk learned from experience earlier that year, when he collaborated with the United Nations on the VR film, Clouds Over Sidra, which takes you inside the life of Sidra, a 12-year-old Syrian refugee. At that point, stories of the Syrian refugee crisis dominated the news, but often failed to reach many Americans on a deeper, more human level. But one thing VR can do that the nightly news can't (not yet, at least) is give viewers intimate access to the experiences of others, creating an immediate, almost disarmingly real understanding of another’s world. It's as close to walking in someone else's shoes as you can get without literally putting them on. 

Milk isn't the only one who believes that in VR, we have the potential to become better listeners, caretakers, and global citizens, using pixels and haptic tech* to tap into a shared universal experience. As enthusiasm for VR’s gaming capabilities wanes, curiosity about its applications to the fields of mental health, rehabilitation, and community-building has only grown. Dozens of projects and research studies currently under development are breaking ground in areas ranging from opioid addiction and substance abuse to physical therapy and PTSD, all of which have the cumulative effect of potentially overhauling the entire field of patient care.

Before VR was a tech conference sideshow, professor Patrick Bordnick, dean of Tulane’s School of Social Work, was exploring its therapeutic uses. Bordnick has studied addiction treatment and VR since the early ’90s, when he created an immersive VR project known colloquially as the “heroin cave,” a series of wall projections set in a sensory-controlled room where subjects could re-enact a series of role-play scenarios aimed at treating substance dependency during early stages of rehabilitation.

The team's custom-designed VR system didn’t work like the Oculus Rift or Gear VR. Instead, it utilized goggles that transformed images projected onto the walls into a 3D HD experience. Once inside, patients could be trained, with the aid of a therapist, to identify and resist triggers. (For example, in one scenario a participant walks through a house party and bypasses a group snorting heroin.) The project is based on traditional cue-reactivity therapy, in which exposure to a predictable trigger activates a patient's addictive behavior hoping that, through controlled exposure, patients could modify their reactions.

“When I started studying why people relapse 20 years ago, we’d bring them into the lab, show movies of users, and monitor cravings and discuss relapse,” Bordnick said. What struck him was the ineffectiveness of the initial approach. In a sterile office or research lab, subjects were far from the triggering context. If he could, somehow, put patients in a digitized version of real-world settings, he felt he might be able to better help them navigate real-world stumbles.

The prohibitive cost of VR headsets, which can run as high as $300 for a basic kit, often put them out of reach for many of his patients. His current goal is to bring his work to smartphone-based reality platforms or design an app that places users in a virtual environment during therapy. He’s also developing a range of environments, from a "heroin shooting gallery" to a cigarette-filled party, for skills-based treatment. “What we were able to show is that six months later, the people who received the skills training had the confidence to use those skills, and were even smoking less."

Another self-proclaimed "grandfather" of VR patient care is Albert “Skip” Rizzo, Ph.D., director of Medical Virtual Reality at the Institute for Creative Technologies, and research professor at the Department of Psychiatry and School of Gerontology at the University of Southern California. There, Rizzo built a VR lab to treat brain injuries and ADHD, later approaching the school’s Institute for Creative Technologies. The Institute, funded by the U.S. Army Research Laboratory, is also home to Light Stages, an innovation incubator that hosts special-effects pros and game developers, and collaborates on immersive simulation technologies for the military. (They also provided many of the visual effects for Avatar.) Rizzo worked with the team to develop Virtual Reality Exposure Therapy, or VRET, and a system called Bravemind.

An “academia-Pentagon venture,” Bravemind uses computer-generated imagery built specifically for veterans from the Afghan and Iraq conflicts to help cope with PTSD and other combat-related symptoms. Scenes include everything from Afghan villages to crowded Baghdad markets, and are accompanied by sensory stimuli like grenade explosions or subtle ground vibrations. So far, the therapy has been used in over 75 sites, including military bases, VA medical centers, and clinics across the United States.

“I was frustrated with the limitations of tools used to help recover cognitive function after traumatic incidents, like a car accident,” said Rizzo. In the early ’90s, he observed that one of his subjects who was unable to stay focused on rehab for longer than 10 minutes, was constantly glued to his Gameboy. After that, Rizzo began to consider gaming as a therapeutic tool to enhance patient motivation and rehabilitation. Ironically, much of the VR therapy Rizzo works with today evolved from Full Spectrum Warrior, a first-person shooter game initially developed as a training tool for the Army and released commercially in 2004.

“From time to time, naysayers will mention VR is dead, only because it hasn’t radically re-shaped the gaming industry in the way it was hyped. But even if all innovation stopped tomorrow, we would be at a sufficient level to continue to do great stuff, clinically," said Rizzo.

Interactive designer Javier Soto Morras, formerly of design lab IDEO and now Berlin's Neeeu Studio, works in VR to experiment and develop environments that adapt to our emotional and psychological states. "One of the hardest things in this field is to understand the effect that design can have on users," he said. "Not just the obvious emotions, but also 'irrational' ones like pleasure, arousal, calm."  

While at IDEO in 2016, he built a platform using Unity** and Arduino*** that enabled him to open the fourth wall of VR, allowing other designers to expand upon his work. His prototype utilized a series of sensors that link the readings to Unity and open-source development hardware, creating an atmospheric space that can change depending on a user’s biosignals and applied pressure. Processing signals from a user’s heart rate, electrodermal activity, or breathing, the code translated those voltage signals into beats per minute, which were then used to update the scenario in each frame. When a heart rate went up, the light in the virtual reality world changed to a soothing blue; when it went down, the illumination flashed and changed to red. The idea behind this particular experiment tested whether changing color or lighting patterns could affect the user’s mood, either calming or further agitating. In another set of experiments, Morras used light and pressure sensors to change entire environments.

"I saw the opportunity to monitor brain activity or pulse and let that data influence the experience inside VR," he said. "Not only could we could begin testing our concepts before building them, we could also make full use of the possibilities of VR to induce moods." Morras believes these enhanced interactions are just small steps, but could ultimately lead towards a more personal, immersive VR experience, with users someday able to create their own unique therapeutic spaces.  

The clinical benefits of VR have not been lost on the commercial sector. OnComfort is an app that offers clinical care to cancer patients, helping to “self-manage stress, anxiety, and pain through evidence-based psychological interventions in VR.” For example, at a hospital, before a surgery or during interventions, doctors can place a VR set on a patient’s head and immerse them in a 360-degree environment “specifically designed for medical indications;" basically, surround-sound, doctor-approved YouTube clips.

This Belgian-based company developed its first VR module in collaboration with students from Houston, testing the alpha on doctors and the beta on patients. Previously, OnComfort co-founder and CEO Diane Jooris delivered "psychological interventions" during procedures and surgeries at MD Anderson Cancer Center in Houston, Texas and observed the impact of these interventions on a patient’s comfort, anxiety, pain, and recovery. "But my impact was limited in terms of time, availability, or even language," she laments. She needed to find a way to conduct these sessions in multiple languages, and without being present.

In studies, OnComfort was able to reduce postoperative anxiety by 56%, postoperative pain by 45%, and save patients 50% of recovery room time compared to a control. Currently, the program is in use in hospitals throughout France, Belgium, Korea, and Australia.

VR is also being used outside of traditional clinical settings. Doctors Without Borders (DWB), who recently experimented with 3D printing and VR technology to design field hospitals, has also been using VR to help train doctors and recruits to deal with potentially devastating real-world scenarios like mass casualties. “Training was the primary purpose for our embrace of VR," says Negin Allamehzadeh, a VR producer for DWB. “It's become a way to help people who haven’t been to the field or dealt with a mass casualty incident get a sense of what it's like, both in terms of the chaos and unpredictability, and the stress of actually experiencing that.” She explained that seeing and feeling events from the perspective of someone who's just been injured are less effective when it’s outside of VR, for example on a computer or by way of a dry powerpoint projection. “You just wouldn't get it as viscerally on a desktop.” (You can still watch the VR documentaries of their progress without a VR viewer, but it goes without saying that the experience just isn't the same.)

Another powerful use of VR in service of greater empathy has been designed by artist Heeju Kim, who attempts to recreate "the experience of autism through design" in her VR Empathy Bridge for Autism Toolkit. Designed to enable people to experience the visual, auditory, and speech differences that come with autism and, hopefully, to engender greater understanding of a sometimes mystifying set of behaviors, it is also surprisingly lo-fi.

“I've been living with my younger brother with autism for more than 20 years," Kim said. "The empathy and understanding of others means the world to the people who have autism and to their families." She explains that many with autism behave differently because they sense the world in such a different way, which can be jarring to others.  

As part of the kit, Kim created a headset with "low arousal colors," or more neutral soothing tones, favored by those with autism, reproducing an "autistic vision" of surrounding stimuli. There's also a set of earphones that provide the overwhelmed auditory soundscape that is one of the most distinctive sensory characteristics of autism. "Every noise and sound is magnified, distorted, and muddled so it becomes very hard to make conversation while using the tool, much as an autistic person would experience life."

So far the kit has been part of exhibitions in Dubai and London and will be featured in shows at the Philadelphia Museum of Art in 2019 and at Walker Art Center in Minneapolis in 2020. But we can easily imagine it being distributed by social workers and hospitals to families whose loved ones who have been newly diagnosed with autism, helping them to better understand the patients unique needs and sensitivities.

There's no doubt that VR has the potential to make a huge impact on the way we approach patient care, helping the disabled, and those struggling with addiction, pain management, and mental illness. Given its demonstrated success, it's not hard to imagine a future in which a VR headset will be provided in every hospital room, and VR techniques included in every doctor's basic training. But behind each of these "empathy machines" is a quest for the authentic and very human response. While VR can never really truly replicate this experience, it has the potential to help us, as a society, get closer to what it means to be humans sharing a planet—and, ultimately, evolve into more compassionate and caring global citizens.

*Haptic tech: a form of kinesthetic communication that duplicates the sensation of human touch via vibrations, visuals, or motions. It is frequently utilized in next-gen gaming, VR, consumer electronics.

**Unity: a game development platform used to build superior 3D and 2D games.

***Arduino: an open-source electronic prototyping platform that allows users to create interactive electronic works.