Hello. My name is David Rhew, and I'm the Global Chief
Medical Officer for Microsoft. I'm a Physician, a Technologist, and a Health
Services Researcher. Over the past 25 years
I've studied how technology can be used to
improve health outcomes, specifically access to care, quality of care patient safety, and ways to improve the patient
and provider experience. I'm excited today to be
able to share with you how mixed reality can be used to transform health care
and improve outcomes. To do that I'd
like to share with you some slides and some videos, so let's just jump
right to that. The real exciting things
about mixed reality is that the imagery and the visualizations gives
you the sense that you are actually there and that
these animations are real.
It allows us to be able to immerse ourselves into
environments and be able to think of ways that
we might be able to interact with each other and
the environment in ways that we've
never thought of. Now, oftentimes this is
viewed in the context of entertainment or fun
and exciting things. But what we've also
seen is that there are several use cases that could be relevant
for both clinical practice education and training. But before we jump
too far into this, I wanted to provide a
disclaimer regarding the things that I'll be talking about as part of
this presentation. I'll be covering both
slides and sharing videos but largely this is about
the art of the possible. These are things that could potentially be done
with the platform. Now, in some cases
we have ISV partners that have built on the
Microsoft platform and they're leveraging the Microsoft
tools to be able to create immersive
experiences that have been validated and approved through regulatory mechanisms and are now marketed for
certain use cases, I'll showcase some of those. But what we also
want to recognize is that there's a desire
for Microsoft to be able to provide the
platform but not necessarily replaced the clinical
decision-making that clinicians have.
We also are looking
to make sure that we are not making any claims
regarding the accuracy, safety and reliability of the technology and the platforms as it pertains specifically to the mixed reality
clinical use cases. Now, with that said,
let's just jump right into it and
let's talk about how mixed reality can empower every individual
and organizations across the planet to be
able to achieve more.
In order to do that let's start thinking about the
paradigm of health care. In fact, there are many
things that we look at healthcare is being
tried and true. These are sayings that oftentimes
had been instilled into our daily practice
one of which is See on, Do One, Teach One. This was coined by Dr. William
Halstead in the 1890s, and it specifically
referred to the fact that clinicians in the course of training need to see several
types of procedures, they need to see several types of patients with
certain conditions.
Then they need to perform
the procedures on the patients and
gain some expertise. Then eventually
they get to the top of the pyramid
where they're able to communicate and teach
others how to do this. As you can imagine, this
is a very narrow pyramid, it doesn't allow us to be
able to scale this and unfortunately that is created problems when we think
of access to care, with virtual reality, with mixed reality,
augmented reality, we have tools now available
to us to be able to create immersive experiences
that can transform the opportunity for providing
education and training. What we want to
also recognize is that the devices
that we now have available that allow
us to be able to change the way that we
see the environment to be able to see three-dimensional
images and be able to understand how these images can interrelate with each other. It allows us to be able
to not only see it but potentially change the way
that we look at the world. This is a quote
from Satya Nadella, the CEO of Microsoft.
When you change the
way you see the world, it change the world you see, and that's exactly
what we're talking about here with mixed reality. In the context of clinical use cases we are
going to focus on three. The first is around holographic
patient consultation. What that really means is
in the course of talking to the patient about a procedure or a particular type
of intervention, we oftentimes need
to communicate information in ways
that go beyond words.
A picture would be great but the three-dimensional
interactive picture that one can look at that
may be specific for them is exactly
what patients need. Now, in the context
of consultation we'd love to be able to bring
consultants in on every visit, impractical to be
able to do that. But with virtual care and with some of the tools
we have now available, we can do remote expert
consultation and have them available all at the time that we're
seeing the patient, and have those consultants
be able to see the same type of things that the individual
clinician is, that's standing right in
front of the patient. In terms of training, simulations that allow
us to be able to understand the
different anatomies, to be able to provide
procedures and enable junior clinicians
to be able to practice in a safe setting but also
teach them the proper way to do things and avoid certain aspects that
could lead to danger.
That's something
that we now have available with these mixed
reality applications. What I'd like to do
is I'd like to start off first talking a bit more about the technology that
underlies a lot of this Now, when we think of mixed reality probably the very first thing we think of is the headset, the HoloLens or the HoloLens 2. That is a core
part of what we're talking about but
the imagery to that, the analysis is actually being
done to the back-end with a powerful platform on Azure
is called Microsoft Mesh, and that allows us to be able to create the
three-dimensional images, it allows us to be able to pull the data from the
different sources and be able to provide that level of interactivity in
immersive nature.
Additionally, Microsoft Mesh interfaces not just with the
HoloLens but a variety of other types of technology
tools such as PCs and mobile devices as well as other virtual reality headsets, so we can create an ecosystem of different types of devices
that can be used by clinicians and patients
as they're immersing themselves or working with individuals that are in
the immersed environments.
Then on top of that
sits the applications. Dynamics 365 has allowed
us to be able to create immersive
experiences that connect us to
different individuals. We now have Microsoft Mesh apps, Altspace VR apps, and of course, partners are building
on these platforms and creating incredibly
immersive and important types of
clinical applications using the Microsoft
technologies. That's what I'd like
to share with you, some of those
partner solutions in the healthcare space
that are really transforming the
healthcare industry. Let's start focusing on
these partner solutions. The first is around holographic
patient consultation. As I mentioned before, what we're doing is we're going beyond simply showing
a picture but creating an interactive tool for both patients and providers
to be able to communicate. In this example what
we're going to be talking about is the
Microsoft HoloLens combined with apoQlar's virtual surgery
intelligence or VSI. Let's take a look at this video to understand a bit
more about this. [MUSIC] Medicine is in the
midst of digital change. Mixed reality and
artificial intelligence are already improving
patient care today. The virtual surgery intelligence or VSI is doing just that.
With this software
solution, MRI, CT and other medical
scans are displayed in 3D and Microsoft HoloLens
2 mixed reality glasses. This gives the patient a deeper understanding
of the results, surgical procedures
and treatment methods. Before a surgery
the doctor uploads the CT and MRI data
onto HoloLens 2. The VSI recognizes the patient
to anatomical landmarks and positions the
stored image on the surgical sites with
millimeter precision. During the after surgery round the patient's
data is quickly retrieved and displayed
using VSI and HoloLens 2. At-a-glance and
without a computer the doctor can concentrate
fully on the patient, answer questions and discuss
follow-up medical care.
Being able to easily dictate
a report, take photos, and to retrieve archived
images saves valuable time, facilitates everyday
life in the hospital, and provides quality assurance. That would also make
other processes in the hospital more efficient. VSI and Microsoft HoloLens 2 define a new medical standard. What we saw there were
two important use cases. The first was around
pre-surgical planning using the HoloLens to help understand exactly
what the procedure would involve and educate
the patient as well. Then the other was after the surgery or after
the procedure, now taking a look at
all the different ways that these technologies
can be used to improve operational efficiency but also imagined being able to
pull down the images from the CT MRI recreate the new structure
and compare that to the old.
These are the opportunities
for how we can apply this in the
clinical setting. One other thing that
we've seen is that with this type of technology
we can also provide consultation where experts
and other clinicians can all gather
around the patient virtually and
provide their input, and this was something
that we had seen, take full during the pandemic, the Microsoft HoloLens
2 in combination with the Microsoft Dynamics
365 Remote Assist enable clinicians from the
National Health Services to be able to take care
of COVID patients. What I'd like to do is
I'd like to be able to show you a video that was provided to us by the NHS
Imperial College Health to be able to understand
exactly how this was deployed.
[MUSIC] Looking at now opportunities
to apply mixed reality in a real-world setting
to be able to provide that level of
virtual collaboration, there's actually been
studies that have been performed on this to show the significant reduction
in PPE as well. In addition to protecting the conditions from
exposure to the virus, what we've also seen is a significant decrease in PPE or personal
protective equipment. One other area is
augmented teaching. This is an opportunity for
us to be able to apply Mixed Reality to
individuals that are in the course of training. It could be medical students, nursing students,
pharmacy students, anyone that needs procedural
training as well as residents and other types of individuals that
are junior faculty.
It provides an
incredible opportunity for us to be able to help us understand
everything from the anatomy to
surgical procedures. Again, this scenario
is a combination of the Microsoft HoloLens
2 technologies with, in this case interactive Kahneman's hollow
anatomy application. This was done and tested at the Case Western
Reserve University. What we saw with the Case Western Reserve
University analysis, was that the students
that we're using the hollow anatomy solution
with the HoloLens 2 had a 50 percent higher
retention rate and 40 percent less
clinical class time, so an extraordinary opportunity for us to be able to enhance and improve the efficiency of
teaching for clinicians. Now, let's get back to what we just talked
about before, the technology that
underlies this. Much of what we're
talking about here is in the context of
Microsoft Mesh. It is on Azure so there's a significant amount of AI
that's been applicable but we've also built on top of that and on top of
that we've created an opportunity for individuals
to be able to collaborate.
Imagine being able to go into
a virtual conference room, have an image of the
particular organ in three-dimensions
in front of you, and rather than all of the
participants being there just through a video conference
their avatars are there, and avatars can interact
with the actual organ, be able to then dissect it, show particular areas, have people look at
different angles. This is what is possible
now with Microsoft Mesh. More recently Microsoft has announced that Mesh can be
launched directly out of Microsoft Teams which
gives us an opportunity to be able to then
leverage the fact that individuals and
organizations are now leveraging
collaboration tools such as Teams for telehealth and virtual care and then launch
into these environments. An incredible
opportunity to provide intersection between the
different Microsoft products, and that's exactly what
Microsoft is doing.
We are taking a look at all the different
technologies that we have available today, everything from Azure
to Teams to Dynamics, and all the platforms that
we have built on top of it including Mesh and
integrating them all. But at the center of all of
this is healthcare data, and if we think about how
healthcare data needs to flow from all the
different systems the great way to
visualize it is with this circle or this
diagram in front of us.
Imagine it being divided
into three parts. In the upper
left-hand part we're now talking about insights and ability to understand
the individual at a very personal level. That means we need to
pull data not just from our environment but also from the electronic
health record, from other datasets
and normalized that through open APIs
and fire standards. In the process of bringing
all these different datasets together on a common
platform we now have the ability to apply
artificial intelligence on top of it and in that context we can apply more than just
simply the context of what we know to
be visualization of the imagery but we also
can potentially apply AI to transform some of the information into
decision support, ways that we can
now act upon it.
Then from there it needs to be collaborated and coordinated, and that is something where the collaboration
tools that we've just talked about can be
incredibly helpful. Part of what we're
now talking about is a transformation of how
virtual care is being combined with
artificial intelligence but what we're also seeing is that there are
other virtual care and AI tools that we have
at our disposal. We have seen chatbots and
the rise of the information, the need for consumers
to be better informed all being brought into a virtual care environment. We've seen the use
of voice technology being converted
from voice to text, understood who's
actually talking and then from there the AI
will allow us to be able to structure that into a clinical progress note which then can be integrated
into the medical record.
This is also something
that has been launched during the
pandemic as part of the ambient clinical intelligence
technologies that are now embedded into Microsoft
Teams for virtual care, we have now an
ability to transform the way that virtual
care is delivered by bringing AI and ML into
the technology suite. Now what we're talking about is an opportunity to re-imagine
what healthcare can be. We're getting better insights by connecting surgeons
and clinicians with the medical data that matters
most, better experiences. Now, students can
perfect their skills by practicing on virtual tools
that are in three-dimensions.
Then providing better
care by allowing us to be able to provide
preoperative planning, post-operative analysis,
and better information that's provided to the patients as part of this
overall experience. Throughout this all what
we are learning is that mixed reality has an
incredible potential to transform the way
that we take care of patients and that we train
clinicians across the world. Thank you very much. [MUSIC].
Free Discount Prescription Drug Cards
