Because Doctors are scientists and scientists are all about identifying, analyzing and solving problems, they are always among the first to harness new technology to improve the health of their patients. With health representing 20% of the U.S. economy and tech over 10%, scale dictates that tiny improvements in either sector have a huge impacts. Slight improvements in training, a slightly better test or tool, or a faster diagnosis, have the potential to save thousands of lives and billions of dollars. As a result, we’re finding a lot of early innovation in new virtual and augmented reality technology in medicine.
We see seven themes emerging from new VR and AR apps for health: (1) training (2) education (3) visualization (4) psychology (5) Telehealth and telesurgery (6) screen consolidation and (7) physical training, health, and fitness. This article will focus exclusively on the applications of Virtual Reality – fully replacing the world with a realistic digital simulation of patient-specific anatomy and pathology.
Augmented Reality, which puts data in the user’s field of view, like a pilot’s “heads-up display” (HUD), has many compelling applications to offer the medical profession. With tips from sources, we found one AR Health startup, Echopixel, in stealth mode that wants to map CT scans on real patients. Doctors are surrounded by screens all day, but they have to look away from the patient to see them. AR headsets solve this problem while opening up whole new areas for innovation and augmentation. A lot of technological change is going to manifest in health care first.
Justin Barad, the CEO and Founder of Osso VR, is a practicing pediatric orthopaedist. In a previous life, he was a game developer. Barad knows firsthand the challenges orthopedic surgeons face, especially when learning new techniques and devices, and he's drawn on his background in interactive media to create important new tools to train surgeons. "We surgeons operate with a fallacy of transferability, which means we come to believe that because we’re good at one kind of surgery, we will be good at another. The data says we are starting fresh, just like a resident, no matter what our experience with other tools and approaches,” said Dr. Barad. This is one of the reasons it’s so hard to introduce new techniques and devices in clinical medicine. It’s too hard and, in some ways dangerous, for a specialist to learn it by doing it.
Hospital workers, long known for graveyard humor, often joke accident victims and the homeless who require emergency surgeries are “MRBs”, short for “Medical Resident Benefits”. Residents are highly educated and observe many surgeries, but they only operate on one cadaver before doing surgery on a patient. Their results 300% worse than experienced surgeons. “In med school, they say ‘see one, do one, teach one’. It’s a confidence builder, but the truth is you need to do 50 to 100 cases for proficiency,” said Barad.
Osso uses virtual reality to enable surgeons to perform realistic orthopedic surgery, replacing an old, complicated, expensive manual simulation created by the aptly named Sawbones Corp. Sawbones provides a realistic molded plastic model of the bone for orthopedic training and practice. The models are expensive and can only be used once. Much of Osso VR’s business comes from medical device manufacturers, who now have a way to show surgeons the benefit of the new implants and techniques while training them in their use. Studies show surgeons who train with Osso's system achieve test results twice as good as good as those trained using current methods.
ImmersiveTouch of Chicago, Illinois founded in 2005, is surgical simulation company that uses Vive and Rift Head Mounted Displays (HMDs), patient-specific anatomy and haptic (tactile) feedback in order to train surgeons and team members, and educate patients themselves. The company provides modules for Neurosurgery, Orthopedic Surgery, Ophthalmology, ENT, and MIS procedures “This technology represents a major shift in how VR technology is delivered to surgeons,” said Jay Banerjee, COO. "ImmersiveTouch training reduced surgical errors by 54% comparing a control group using current training methods." Dozens of elite medical and educational institutions including Cleveland Clinic, Johns Hopkins and the University of Chicago are currently using ImmersiveTouch to train their residents with procedural modules using a library of generic cases.
“We are the first company in the nation to integrate VR based surgical training with haptic technology. This allows your hand to feel the resistance of a surgery and be able to tell the difference between skin, muscle, and bone while working on a 3D human patient," said Pat Banerjee, Ph.D., CEO of ImmersiveTouch. “The Mission Rehearsal virtual reality environment is a vast improvement over the distortions of x-rays and other traditional imaging that can misrepresent the challenges inherent in the patient’s anatomy."
Surgeons at the All India Institute of Medical Sciences (AIIMS) in New Delhi recently used the ImmersiveTouch Mission Rehearsal virtual reality surgical platform to prepare for an extremely rare surgical procedure to separate Siamese twins co-joined at the head. Using VR and haptic robots that feel like a patient, surgeons were able to immerse themselves in a 3D virtual operating room. “We could see, feel and study connected tissue, discuss anatomy, and examine surgical pathways to practice the surgery multiple times and plan the most effective surgical roadmap,” said Dr. Deepak Gupta, who led the surgical team of 40.
A Cleveland, Ohio based startup, Surgical Theater is transforming the way surgeons consult with patients, plan complex surgical procedures, and use medical imaging to navigate during surgery and post-operative review. Their SuRgical Planning (SRP) technology is the first and only patented and FDA-cleared platform for neurosurgical pre-operative planning. SRP uses a patient’s own CT or MRI to create a dynamic 3D model the surgical team can then use for pre-operative planning.
Using flight simulator technology developed by former Israeli air force officers Moty Avisar and Alon Geri, Surgical Theater enables neurosurgeons to “fly” through the scene of the mission – and even rehearse complicated procedures – before operating. "Surgical Theater technology gives you an amazing ability to immerse yourself in the anatomical structures and mentally rehearse the entire operation ahead of time. It also allows you to reorient yourself during critical steps of a complex surgery,” explained Dr. Neil Martin, Chair of the Department of Neurosurgery at UCLA Medical Center. The Global Virtual Patient Simulation Market is poised to grow at a CAGR of around 19.6% over the next decade to reach approximately $1.49 billion by 2025, according to "Virtual Patient Simulation Market Analysis and Trends- Technology, Product - Forecast to 2025" a market research report by market analytics firm Research and Markets of Dublin.
Given the remarkable progress, clinical applications of VR are already making, it is no surprise to see its use dramatically increasing in medical schools. University of California, San Franciso, medical school is using Organon for the HTC Vive. In early 2017, Colorado State University in Fort Collins, Department of Biomedical Sciences created its own VR app to visualize and manipulate magnified models of the human brain and nervous system in all dimensions. The team is now working to expand the program to include all the structures of the body.
"Cadavers and textbooks have limitations in what they can convey. Virtual reality improves upon learning from a cadaver by going from the skin layer all the way down to the bones and back again. Every single layer can be moved independently allowing students to see the relationship between the muscles and nerves and organs, zooming in to the microscopic level if desired. It’s a learning experience almost like putting a puzzle together,” said Assistant Professor of Anatomy Derek Harmon, Ph.D.
Like Organon, The Body VR, best known for the popular free VR app "Journey Inside a Cell" (on Steam for Rift and Vive), which has 500,000 downloads. The "Anatomy Viewer" brings the patient's MRI and CT Scans into VR. The Body VR's core business, however, is the highly specific app "Colon Crossing", now the subject of a clinical study with several participating hospitals. Mo Ben, co-founder, and CTO told me "it's early, but the medical field is receptive to disruptive new technology. They see how flexible, personal and inexpensive it can be. These are people who pay fifty thousand dollars to make a video." Mo said the company envisions a digital health care system in which a large, layered 3D view of the body is the ideal interface for our personal health care records.
Dental Schools have been using Virtual Reality for decades. Dental simulations do not require an HMD, but are like those for airline pilots, which cost hundreds of thousands of dollars. The dental simulators feature real instruments and robotic feedback. Dental students stand over a model and look at an adjacent screen for a photorealistic real-time 3D image of the procedural simulation underway. VR could reduce the cost of these simulators dramatically.
The veterinary profession is also looking to VR for its educational needs. LlamaZOO Interactive, in partnership with leading international veterinary universities and anatomists, has created JetsonVR, the world’s first virtual reality canine dissection software, built from MRI & CT data. A somewhat cruder version of Canine Anatomy was presented by Texas A&M at SXSW in 2016.
Virtual Reality applications are saving lives in hospitals around the world today, and gaining more converts among decision makers. At the same time, the next generation of doctors, those entering medical school today, are the first digital natives, born with smartphones in their hands. Like former game designer, orthopedic surgeon, and founder of Osso VR, Justin Barad, these young doctors are bringing their technology with them, expecting it to problems, and will make their own solutions when they don't.
This is the first in the three-part series about VR and Healthcare. The next stories will focus on VR’s use in clinical psychology and telemedicine.
