A few years ago, augmented reality belonged to the realm of fiction. We could see it in movies, but at the time, the concept was too futuristic or too fictional to give a hint of realism or hope of seeing it in practice. Also, very few people realize that the film introduces augmented reality (AR), when they get to the sequence in question. So it wouldn't be surprising if a lot of people were skeptical when told that augmented reality is now developed enough to rival science fiction, let alone in the operating room.
From AR to MR to conquer surgery
While there are several applications of augmented reality in healthcare, its adoption in surgical practice remains one of the most impressive. Indeed, technological advances in scanning and tracking have opened up possibilities to better pinpoint effects according to the user's spatial perception. A precision that is imperative in the world of medicine, as it is not for nothing that we speak of surgical precision.
On the other hand, the technology offers a better interaction, which gave birth to Mixed Reality (MR). This term is more appropriate as it is no longer about adding or augmenting virtual elements to the real world. In fact, we are talking about a real mixing of the two realities where the virtual and the physical interact as part of a single existence. This interaction is also the point that sells MR in surgery. In addition to very precise effects, surgeons need these virtual augmentations to be dynamic and to follow them during the operation without any additional actions than the usual gestures in the operating room or in daily life. Demands that have been met by mixed reality glasses, such as Hololens.
A technology whose advantages cannot be ignored
The use of mixed reality, especially through mixed reality glasses, has allowed the surgeon to better tackle the operating table. The change that will have the most impact on the medical field, with the introduction of augmented reality, is the 3D modeling applied to medicine.
The surgeon no longer needs to estimate the depth or size of an element via one or more images, he can look at it directly in relief, rotate it, zoom it... leaving less room for trials and errors. Moreover, the models are projected directly onto the glasses, which frees the hands, while avoiding the surgeon having to constantly turn his head to look at the monitors next to the operating table. We must also mention the functionality of superimposing the 3D model with the patient himself in real size, in order to offer the surgeons the equivalent of an X-ray vision in real time. Moreover, the model is fixed at the same point in space, offering the possibility to other surgeons to see how the operation is going on via the glasses. Another use case is the guidance function that shows where to perform which operation and with which tool. An assistance that new surgeons will particularly appreciate, although experienced surgeons may consider it somewhat dispensable.
Medicine and mixed reality, a partnership full of promise
These few contributions of mixed reality are already great innovations for the practice of surgery. However, the changes it has initiated in the field of medicine do not stop there.
One application of mixed reality in medicine is in education and training. The 3D models of the different organs and body parts are very accurate (digital twin), allowing students, future doctors, to become familiar with them. In addition, MR offers the possibility to operate on a mannequin as if it were a real person with organs, blood vessels and muscles where they are needed (virtually). Finally, virtual assistance can be used to teach the correct gestures and procedures to follow during an intervention.
Another use case of mixed reality concerns the combination of man-machine. This usage calls upon machines, piloted by a surgeon, to perform complex surgical operations. It is very interesting for very small scale interventions, the robotic arm being very small and just as agile as a human arm or hand. The surgeon performs the operation wearing a virtual reality headset whose images come from cameras mounted on the robotic arm. Thus, the surgeon feels reduced to a very small size with his hands and arms now replaced by their robotic counterparts. However, it is important to note the nuance: this is not virtual reality, but mixed reality, as the surgeon does not move in a totally virtual world but inside the patient's body which is real.
The adoption of augmented, virtual and mixed reality in all professional fields is going very well. With achievements such as successful knee and spine surgeries to its credit, it won't take long for mixed reality to move from "optional" to "essential" status in medicine. Furthermore, the adoption of virtual reality in the field of video games is very fast, although it does not surpass that of augmented reality with social networks. It is therefore important to become familiar with this technology now, which will soon be a general trend in all fields.