Vision is the main source of information
about the surrounding world, a person receives through the visual analyzer
(eyes) more than 80% of conscious information [1]. For example, medical
students in their initial courses are taught about the normal anatomy and
physiology of the human body. For this purpose they hold lectures and seminars
with the use of photos, videos, visual aids and materials. Students need to
understand the mutual arrangement of organs and tissues, but this often causes
difficulties for students when using flat images in textbooks and atlases (see
Fig. 1, 2).
Fig.1. Illustration from a textbook for medical
students[2].
Fig.2. Illustration from anatomical atlas[3].
To improve
students' perception of the taught material, they also conduct classes in the
anatomy room or, for example, use visual models made of plastic (Fig. 3).
However, problems often arise. For example, teachers can have a hard time
teaching large numbers of students in the anatomy lab when every student needs
to see the object of study, and the plastic models sometimes cannot show the
relationship of anatomical structures in all its complexity. In addition,
plastic models are often structurally complex and their parts often get lost or
fall into disrepair.
Fig.3. Model of the skull [4].
After receiving
a diploma, a physician must specialize and renew his or her qualifications
every 5 years. Throughout his or her professional career, the doctor must
attend training courses and keep his or her knowledge up to date. Often, training
courses include video broadcasts from an operating room, which, for obvious
reasons, cannot accommodate all trainees. Similar difficulties arise at
international medical conferences with a large number of participants from
different countries.
However, the
current level of development of visualization tools and algorithms allows to
use computer technology in medicine to solve such problems.
Introduction of
stereoanimation - a method of visual presentation of information in volumetric
form - opens up promising opportunities in the field of medical students
training, postgraduate training of doctors, conducting remote medical
consiliums and even conferences. The use of stereo-animation is a new level of
mutual understanding between a doctor and a patient at the stage of
preoperative preparation and planning of surgical treatment.
Volumetric
visualization will allow to understand better the relative positioning of
organs and tissues, thus avoiding the disadvantages of flat illustrations in
textbooks and atlases. In some cases it is impossible to show anatomical
structures in full size on plastic models, which can be easily solved by the
scalability of virtual images. With the help of stereo-animations one can
clearly show deep anatomical structures which are difficult to highlight on anatomical
preparations (cadaverous material). Another advantage of stereo animation is
that there is no need to create anatomical halls for demonstration and no need
for specialized rooms for storing preparations.
The presence of
equipment that creates a realistic three-dimensional image gives students the
opportunity to better assess the proportions and relationships of objects in
the operating field, which, for example, in dentistry and maxillofacial surgery
is a decisive criterion, both for restoring a tooth with a filling, and when
performing surgical movement of the jaw.
Also, in recent
years, at international conferences it has become increasingly common to see
reports without a speaker in the room: the content of the report and the
speaker's video broadcast are displayed on screens that play back images in
real time in parallel. It is difficult to overestimate the presence of
realistic three-dimensional stereo image of some parts of the medical report in
such presentations.
The
maxillofacial region has a complex anatomical and topographic structure, and
many methods for planning and visualizing treatment results for patients with
maxillofacial defects and deformities have been developed to date, but planning
reconstructive surgery in this area is still a non-trivial task.
The planning and
execution of treatment in the maxillofacial region often requires the interaction
of physicians from different specialties. For example, reconstructive surgery
in the jaw region requires stepwise involvement of an orthodontist and a
maxillofacial surgeon, who may be hundreds or even thousands of kilometers
apart. The use of volumetric imaging would radically reduce the number of
patient transfers from one locality to another, i.e., be used as a means of
telemedicine.
Reconstructive
surgeries of the facial skeleton are often performed not only for functional, but
also for aesthetic reasons (to eliminate a hooked nose, reduce the size of the
lower jaw, increase the cheekbones), and the doctor and patient sometimes need
a fair amount of time to come to an understanding of the compromise between the
patient's wishes and medical capabilities. Today, doctors for this purpose draw
pictures on paper, create temporary structures on the teeth, use computer
modeling in specialized software, and make stereolithographic models for
treatment planning. Despite the use of all methods of visualization of the
planned treatment, the patient may be dissatisfied with the aesthetic result.
In such cases, stereo imaging can provide an objective tool for evaluating the
treatment.
Thus, the
application of stereoanimation is a new and promising method of visualization,
which allows to increase the efficiency of training of medical students,
interns, residents, and doctors undergoing advanced training. This method can
also have clinical application in aesthetic surgery both for planning of
surgeries themselves and for presentation of a visual treatment plan to
patients.
It should be noted
that the practical application of the joint work of scientists, specialists in
various fields of science, such as, for example, medicine, computer graphics,
mathematical modeling, is quite in demand and is of great importance. For
example, the algorithms and practical application of mathematical methods in
dentistry are described in [5-8].
In this paper,
we consider the possibility of applying three-dimensional volumetric imaging in
maxillofacial surgery using modern imaging systems to practically solve the
problems described in the previous chapter.
Wide spread of
modern systems allowing to visualize objects in volumetric view has generated
great interest to development of practical algorithms of stereo representation
of static and animated images which is reflected in works [9-12]. Nowadays
stereo visualization arouses interest both in entertainment sphere (3D movies
in cinemas on a big screen, widespread distribution of household smart TVs
allowing to watch 3D movies using special glasses at home) and in scientific
sphere using modern computer systems and technologies.
Note that quite
a number of computer programs used by doctors, for example, to visualize the
results of computed tomography of patients, allow representing research objects
in three-dimensional stereoscopic view (i-CAT Vision [13], OnDemand3D Viewer
[14], RadiAnt Dicom Viewer [15]). Thereby confirming the importance of such
representation in medicine. However, all these programs use rather primitive
methods of volumetric visualization using the anaglyph method [16]. Using
anaglyph requires practically no special equipment other than cheap, readily
available anaglyph glasses (Fig. 4) and allows the use of an ordinary computer
monitor.
Fig.4. Cardboard anaglyph glasses.
In this case,
the image displayed on the screen when observing without such glasses looks
blurred (Fig. 5).
Fig.5. Example of an anaglyphic image.
But the use of
anaglyph method has low quality of visualization, and has other significant
drawbacks, which is problematic for practical application in medicine to solve
the problems described above. The main disadvantage of the anaglyph method is
its unsuitability for visualization of color images. In this case after a
short-term stay in anaglyph glasses an observer's color sensitivity decreases
for quite a long time, a headache occurs and a feeling of discomfort from
perception of the usual world appears.
The use of
modern systems for representing objects in the volumetric view allows to solve
the problems described in the previous chapter, devoid of such drawbacks, and
even there are devices that allow displaying visualization in the volumetric
view without the use of additional devices such as stereo glasses or helmets.
Thus, in works [17-18] in particular described methods of developing algorithms
for volumetric visualization of results of mathematical scientific calculations
on such devices as autostereoscopic monitors. Using autostereoscopic monitors
not only to visualize the results of scientific calculations, mathematical
modeling, but also in medicine, in particular in maxillofacial surgery,
dentistry and other areas of medicine, is, according to the authors, an
important and promising direction, allowing to solve the problems described
above in practice, when used to teach medical students, postgraduate training
for doctors, conducting remote medical consultations, conferences and as a tool
of telemedicine.
The principle of
operation of autostereoscopic monitors is described in sufficient detail in [17].
The technical side of the principle of autostereoscopic equipment is the use of
parallax baffles or Fresnel lenses, installed behind the protective glass of
the screen (Fig. 6).
Fig.6. The principle of operation of the
autostereoscopic monitor.
Such monitors
allow viewing stereo images by providing several fixed segments in space for
observation, and the viewer can move from one segment to another, getting the
opportunity to view the demonstrated object in 3D from different viewing
angles. When the observer's head is in a certain position in front of the
autostereoscopic monitor, his right and left eyes receive different images
(stereo pair). In this way, a convincing illusion of 3D depth is created. At
the same time, as mentioned above, there is no need for viewers to use any
additional devices for observation in the form of special stereoscopic glasses.
The most
important feature of the autostereoscopic monitor is the ability to demonstrate
the visualization object using a composite frame containing views of the
visualization object at different angles, forming a specific sector of view
(Fig. 7).
Fig.7. Composite frame of autostereoscopic monitor for
visualization of CT scan results.
It should be
noted that this type of representation of visualization object in the
volumetric view on autostereoscopic monitor screen provides maximum quality
compared to conventional stereoscopic representation, not to mention the
anaglyph method, allowing to view objects in a sufficiently wide sector of
view. These nine views form eight stereo pairs ([1|2], [2|3], [3|4],..., [8|9])
and an observer can view only one of the stereo pairs depending on their
position in a particular sector of view. Moving from sector to sector, the
observer receives volumetric information about the object, using all nine
perspectives, that is, as if looking back, for example, the results of computer
tomography from different sides.
It should be
noted that so far we have described advantages of visualization of above
described example of computer tomography on autostereoscopic monitor screen
only for static image. But invaluable contribution is creation of
stereo-animations for displaying on the screen of autostereoscopic monitor for
practical application in medicine, i.e. creation of full multiframe video
films, each frame of which is composite, consisting of 9 kinds of object. At
the same time the operator of the stereo unit can pause the animation demonstration
at any moment (e.g. for comments of the lecturer during demonstration, at the
request of viewers or giving viewers an opportunity to "look around"
an important detail of the demonstrated organ from different points of view
when moving viewers to different viewing sectors in front of the plane of the
monitor).
At the same
time, there is a problem of obtaining and processing the huge amount of data
needed to build such animation. For example, a normal movie in PAL video format
implies using 25 frames per second, and even 30 frames per second in NTSC video
format. In addition, it should be remembered that, as mentioned above, each
frame for an autostereoscopic monitor consists of nine ordinary frames, but
each of these nine ordinary frames is a new kind of object. As a result, even
when creating a one-minute video film for the autostereoscopic monitor in PAL
format we need to have 13500 frames.
But, as
indicated in [18], the authors have found a solution for a similar problem
arising in the visualization of the results of scientific calculations and
called by the authors "method of camera flying" around the object.
This solution consists in the use of angular stereobase (Fig. 8) and the method
of reuse of already obtained frames.
Fig.8. Angular stereo base.
Indeed, in the
case of a one-degree angular stereobase and with a successive displacement
angle around the OZ axis, we obtain that the first frame consists of a sequence
of perspectives with a successive change of the camera angle:
the second frame
will look like this:
accordingly, the
third frame will look like:
and so on. The
last frame having the form:
will allow to
loop the sequence of views, and having only 360 monoframes, it is possible to
obtain any long rotation of imaging object in multiview representation on
autostereoscopic monitor screen. Of course, as our experience [17-18] shows,
360 frames is not enough to obtain a high-quality volumetric visualization, and
we used in practice a finer breakdown of the full rotation of the object around
the axis, but the principle of camera overflight remains the same.
This solution is
also applicable in the case of creating multiview stereo animation for
autostereoscopic monitors for volumetric visualization of medical data. For
example, creating a stereo animation of the results of computed tomography of
the patient's head in maxillofacial surgery.
Fig.9. Computed tomography of the maxillofacial
region.
The result of
computed tomography is essentially a set of graphic files, each of which
represents a view of a "slice" of a patient's organ. There are quite
a number of software tools used in medicine that allow transforming these
slices of CT results into a three-dimensional model to be used in CAD systems
(Visurgery [19], Implastation [20]). Thus it is possible to obtain a certain
view of an object from a given point of observation, with possibility to save
this view as a certain part of a composite 9-view frame for stereo-animation.
Of course, it is
impossible to apply this camera-overlapping solution to all practical
applications of this technology. So, quite often it is necessary to create a
volumetric animation for gradual enlargement of a particular local area of the
patient's organ in order to consider the features of this area in detail. In
such cases, the solution described above is not applicable, but the authors
have created an additional set of software tools to automate the process of
creating the required set of composite multispecies frames. The use of the
created tools does not significantly reduce the time required to calculate the
composite frames, as with the camera overflight method, but it does not require
manual intervention and control.
The authors
investigated the possibilities of practical application of stereo-animation
technology for several real cases in maxillofacial surgery. A Dimenco DM654MAS
autostereoscopic monitor [17] inKeldysh Institute of Applied MathematicsRussian
Academy of Sciences, was used as a volumetric visualization device. The monitor
was used in the imaging mode using a composite frame containing views of the
imaging object at different angles, providing maximum quality.
Real results of
computed tomography of several patients were used as test imaging objects. When
creating stereo-animations both camera overflight method and specially created
software tools for automation of creation of multidimensional composite video
frames were used.
In most cases
(Fig. 10, 11) we were able to successfully apply the method of camera
overflight to create volumetric visualization of CT scan on autostereoscopic
monitor screen. Figure 10 shows the results of computed tomography of
craniofacial part of a patient visualized in volumetric form on the screen of
autostereoscopic monitor. Figure 11 is a composite frame of such imaging.
Fig.10. Volumetric stereo-animation of computed
tomography of a patient created using the camera flyover method.
Fig.11. Example of the resulting composite frame.
In this case,
most often there was no need to increase the individual areas of the
visualization object, which can significantly reduce the time to calculate the
composite animation frames.
In other cases,
when a more detailed study of individual areas of the organ of the object of
medical research was required, a set of software tools created by the authors
was successfully applied (Fig. 12, 13, 14).
Fig.
12. Volumetric stereo animation visualizing the patient's frontal bone defect.
Figure
13: Volumetric imaging of a Teflon implant to correct a defect, reverse side.
Figure
14: Volumetric imaging of a Teflon implant to correct a defect, frontal side.
Figure 12 is a
volumetric imaging CT scan of a patient with a frontal bone defect. Figures 13
and 14 are volumetric renderings of a Teflon implant created to correct the
defect on the dorsal and frontal sides.
The use of
stereo-animation is a new and promising method of visualization, which allows
to increase the efficiency of training of medical students, interns, residents,
and doctors undergoing advanced training. This method can also have clinical
application in aesthetic surgery both for planning operations and for
presenting a visual treatment plan to patients. The use of modern devices for
three-dimensional visualization, such as autostereoscopic monitors, which
provide the highest quality of presentation of graphic information in 3D,
significantly improves this process and ensures comfortable perception of
visual information by specialists.
The algorithms
and software used by the authors to create stereo animations in maxillofacial
surgery on stereoscopic monitors can also be used in other fields of medicine.
[1] Physiology of sensory systems: teaching
aid. Eds: Kayumova A.F. et al. Edition «BGMU Roszdrav», 2011, P.115. [in Russian].
[2] Operative
surgery and topographic anatomy. Eds: Kovanov V.V., 4-th edition. —
Ìoscow: Medicine 2001.-
408
p.
[3] Sinelnikov
R.D, Sinelnikov Ya.R. Atlas of human anatomy. Ìoscow: Medicine 1996. -320 p.
[4] Model of the
skull.
https://med-maneken.ru/infusions/al_shop/asset/goods/1211.jpg
[5] V. A. Knyaz, A.V. Gaboutchian.
Photogrammetry-based automated measurements for tooth shape and occlusion
analysis / The International Archives of the Photogrammetry, Remote Sensing and
Spatial Information Sciences, Volume XLI-B5, 2016.
[6] Muraev A.A.,
Dymnikov A.B., Korotkova N.L., Kobets K.K., Ivanov S.Yu. Planning Technique in
Maxillofacial Plasty. / Modern Technologies in Medicine. 2013;5:3:57-62.
[7] A.V.
Gaboutchian, V. A. Knyaz, N.A. Leybova, G. Petrosyan, H. Simonyan, S.V.
Vasilyev. Application of photogrammetric techniques in palaeoodontological
studies trough automated digital shape analysis of human teeth / The
International Archives of the Photogrammetry, Remote Sensing and Spatial
Information Sciences, Volume XLII-2/W12, 2019
[8] Armen V.
Gaboutchian, Vladimir A. Knyaz and Dmitry V. Korost. New Approach to Dental
Morphometric Research Based on 3D Imaging Techniques. Journal of Imaging, 2021,
MDPI, Basel, Switzerland.
[9] Andreev S.,
Filina A. Using stereo presentations for visualization of scientific
calculations results / Scientific Visualization. 2012. v.4. ¹ 1. p.12-21
[10] Andreev
S.V. [et al] Modelling and visualisation of blade assembly with complicated
shape for power turbine / Scientific Visualization. 2015. v.7. ¹ 4. p.1-12
[11] Mezhenin
A.V., Tozik V.G. 3D Visualization using the stereo image effect // Proceedings
of the 2-nd international conference «3D visualization of scientific, technical
and social reality. Cluster technologies of modeling» Izhevsk. 2010. [in
Russian].
[12] Hardware-software
complex of 3D presentations based on a virtual studio and virtual environment /
Vandanov V.G. [et al] // Proceedings of the 1-st international conference «3D
visualization of scientific, technical and social reality. Cluster technologies
of model-ing» Izhevsk. 2009. P.73-77. [in Russian].
[13] i-CAT
Vision
https://kavoimaging.com/en-us/i-cat-flx-v-series
[14] OnDemand3D
Viewer
https://www.ondemand3d.com/pages/ondemand3d/Packages/project-viewer
[15] RadiAnt
Dicom Viewer
https://www.radiantviewer.com/products/radiant-dicom-viewer-standard/
[16] Anaglyph 3D
https://en.wikipedia.org/wiki/Anaglyph_3D
[17] Andreev
S.V., Bondarev A.E. The construction of stereo animations on modern stereo
devices of active and passive types / Keldysh Institute preprints. 2018. ¹ 156.
19
ñ. doi:10.20948/prepr-2018-156
[18] S.V.Andreev,
A.E.Bondarev, V.A.Galaktionov, N.A.Bondareva (2018) The problems of stereo
animations construction on modern stereo devices. Scientific Visualization
10.4: 40 - 52, DOI: 10.26583/sv.10.4.04
[19] Visurgery
https://www.visurgery.ru/
[20]
Implastation
https://ru.implastation.com/downloads/
