Mixed Reality and the Future of Healthcare

The healthcare field is constantly being changed by new drugs, new studies, and new therapies

However, the field often lags when it comes to adopting new technology, and even making the seemingly straightforward move to electronic records has proven to be a lengthy process. Still, new technology not created exclusively for medicine is coming, and mixed reality devices in particular are becoming a reality for many medical professionals and healthcare centers.

Mixed reality combines virtual reality elements with human vision. Head-mounted devices use clear screens to give users an unobstructed view, but various technologies can be used to project images onto the screen. For doctors, MR provides a means of viewing images and data far more convenient than charts or screens. Furthermore, MR can provide new ways of interacting with patients by projecting information onto medical charts or even directly on the patient. As medical schools and other organizations continue to explore MR, experts will devise novel uses for MR technology.

Among all the MR devices coming to market, the one that’s garnered the most attention is the HoloLens from Microsoft. The head-mounted device is more bulky than Google Glass but it offers far greater capabilities by using holograms to create realistic images. HoloLens is also more aware of what the user is seeing, and this greater flexibility provides a host of new use cases traditional AR technology can’t match. HoloLens headsets aren’t cheap, as they currently have a price tag above $3,000, but their cost is relatively low compared to many common medical devices and no doubt cheaper headsets will come to the market in the coming years.

Having an intimate knowledge of human anatomy is crucial for medical students. While charts and interactive computer programs can be valuable tools, medical students often work with cadavers. With MR, students can receive a similarly detailed experience at any time. Furthermore, MR technology can let students zoom in on particular segments, providing a way to explore that’s impractical with a cadaver. Already, medical schools are looking to turn to MR as a primary means of educating future doctors.

Instant Access to Information

Hospitals and medical clinics are becoming more connected, and sensors are ubiquitous in modern medicine. However, doctors still often rely on older technologies when interacting with patients, and many end up reading paper charts to get an overview of a patient’s condition. MR headsets can detect patients and instantly providing relevant medical information to doctors, saving time during interactions and allowing doctors to more quickly respond to emergencies. Simply being able to see a patient’s vital signs without having to read screens or pull out paperwork can save valuable time and allow for more convenient patient interactions.

We recently spoke to Sirko Pelzl,  the CEO at apoQlar, creators of an MRI rendering app for HoloLens

For budding surgeons, being able to view a surgery is essential, as no amount of reading or studying can replace seeing surgeons in action. However, finding time to observe a surgery in progress can be difficult, as space is limited. With MR technology, surgeons can stream their actions live, greatly expanding their audience. Furthermore, surgeries can be recorded routinely, with surgeons saving those that were noteworthy in some way. Surgeons share their techniques with each other, and their experience helps hone the art. With routine recording, surgeons will be better able to collaborate and develop new techniques.

Better Imaging

The benefits of MR extend into offices. Professionals often view medical scans on computer screens and use a mouse and keyboard to manipulate the image and zoom in on certain areas. MR technology can track where a user is looking and respond to gestures, providing a more natural way to analyze an image. Furthermore, many modern imaging processes create 3D images. Through MR, users can visualize depth in a seamless manner. Even more mundane tasks can be aided by MR technology. Loading and modifying electronic medical records can be a somewhat cumbersome process, but new means of interacting enabled by MR can save time.


CT scans are often a significant source of distress for patients, as the noise and enclosed nature of machines can lead to claustrophobia. Through MR and other technologies, medical experts can provide a simulation to help patients know what to expect. Furthermore, MR, along with AR and VR, can be used to help patients relax or distract themselves while being scanned. Simple being able to watch a movie or play a simple game can help patients pass the time and remain still while lengthy scans are underway. These benefits can improve overall medical treatment, as patients sometimes skip medical sessions and may decline helpful tests or therapies due to discomfort. Improving compliance is a powerful tool for improving patient outcomes.

Streamlining Care

Receptionists, nurses, doctors, and other professionals need to coordinate with each other in hospitals and clinics. Working as a team can be a challenge, and professionals often rely on multiple devices for communication and recalling charts and other data. By standardizing on mixed reality devices, health centers can provide a seamless means of communication and ensure everyone can send and receive notes instantly. Furthermore, MR devices can record and share voice communication, making it quicker and easier to send voice notes that can be heard between visits to patients. With MR technologies, health centers can allow healthcare professionals to spend more time with patients.

Many modern VR and mixed reality devices have a battery life of approximately three to four hours, so doctors would likely need to swap batteries or devices during long shifts. However, this problem will no doubt improve significantly in the future, and the technology will become far cheaper over time. Regardless of the limitations, however, MR is already is use around the world for a range of medical tasks. As medical professionals become more familiar with the technology, patients can expect to see headsets in hospitals and clinics on a regular basis in the near future.