1. Introduction

Nowadays, stereo animation begins to play an important role in displaying the obtained scientific results in various applied scientific research and industries. This is largely due to the fact that a situation often arises when a flat image of an object does not have the entirety of information to achieve its goals and limits the analysis to a schematic image or a truncated viewing angle.

Unlike a two-dimensional flat image, three-dimensional stereo allows you to more effectively explore and model objects with a developed spatial structure or interpret photographs taken during the experiment from different angles at given times and combined into animation.

A lot of work has been devoted to this topic. They discuss methods for constructing images for various types of stereo devices, both passive and active types. The difference between these concepts is that passive ones allow one or another material to be demonstrated according to a predetermined scenario, however, the viewer is not able to influence the display process. Active (or interactive) installations mean that the viewer has the opportunity to work with the demonstration material and directly influence the demonstration process. General aspects of image construction for various types of stereo units are presented in [1].

The specific problems that arise when using a computer system to generate and visualize a composite multi-screen stereo frame, and methods for solving such problems are described in sufficient detail in [1-3].

The gradual spread of modern stereo systems has generated great interest in the development of practical algorithms for stereo presentation of static and animated images, which is reflected in [4,5 - 10]. These works present the tasks of constructing stereo images in various fields of research: from the explosion of a Supernova to the processes of oil displacement from porous media.

The spread of stereo animations and virtual reality also revealed a rather important particular structural subproblem. The standard procedure for a report, presentation of something or a virtual walk implies the presence of not only a visual type of information in them, but also a symbolic one - letters, numbers, symbols, metric quantities, names, numbers, state standards, without which the perception and assimilation of information by the viewer is significantly complicated.

The problems of constructing stereo images for geometries and fields of physical quantities have been developed in sufficient detail, but quite a few works have been devoted to the development of specific practical approaches and algorithms for constructing stereo representations of texts and formulas with sufficient expressiveness and the necessary stereo effect. One of them, which cannot be ignored, is the work [11] describing a study conducted in Japan of the perception of stereo images of fonts on the screens of stereoscopic mobile devices.

This paper continues the research [1-3,7,12-14] carried out at the Keldysh Institute of Applied Mathematics RAS. The aim of research is to create effective technology for stereopresentations of scientific results. The research is based on the available stereo units of two types.

The first type of device is a 3D projection stereo system for displaying stereo presentations, educational applications, graphics and films. It is an example of a classical stereoscopic system using two projectors, a screen and linear polarization glasses. The articles [12–13] describe in detail studies on the presentation of textual information by this type of stereo unit using a linear stereo base.  In these works, test inscriptions were constructed with variations of the font, background, font embossment, the whole angle of rotation of the inscription, and shift along the linear stereo base. The parameters that provided the maximum effect were found and the basic requirements for the fonts used and a number of conditions were identified, the satisfaction of which is necessary to achieve the optimal result.

The second type of device is a Dimenco DM654MAS autostereoscopic monitor. Autostereoscopic monitors provide stereo images without having to track the position of the observer. Typically, such monitors make it possible to observe stereo images, providing several fixed segments in the space for observation. The viewer can move between segments, getting the opportunity to view the displayed object in 3D from different angles. The principle of operation of the autostereoscopic monitor is the use of parallax partitions or Fresnel lenses installed behind the protective glass of the screen, which gives it one of the most important advantages: displaying the image does not require special glasses or other devices from the viewer.

An autostereoscopic monitor is capable of demonstrating a visualization object using two methods: either using a composite frame containing views of the visualization object at different angles that form a certain viewing sector - this is called a multi-view, or using depth maps.

In multi-view image construction, nine frames are combined into one image according to the principle of a 3 × 3 matrix. In the first and last (ninth) frames, the desired object, for example, the inscription, is in its extreme positions, and in the intermediate frames it is rotated sequentially by a given angle.

When using the method of depth maps, a certain range of depth is selected, which is manually set by the user by fixing the nearest and far planes. The nearest plane is assigned a value of 255, and the farthest - 0. Information describing the depth is explicitly placed in the image and recorded in the form of gray gradations. The intensity value is thus encoded with eight bits in the range [0, 255]. Further, objects located in a given range fully use the stereo effect depth range that the monitor simulates. This method is described in detail in [9].

2. Some previous results

In previous works devoted to stereo imaging of textual information [12-14], some computational experiments were described. These experiments were carried out for stereo systems of both types — classical stereo installations using special glasses and an autostereoscopic monitor. For the autostereoscopic monitor, experiments on constructing stereo images of texts were carried out using depth maps as well as using a multi-view presentation.

One of the tasks when conducting experiments using a multi-view was that the viewer did not see the transition between each of the stereopairs when changing the viewing angle, and due to this the most powerful reality effect would be created. If the linear and angular shift parameters are too large, artifacts arise: the observer has a feeling that the image in front of him is blurred at the transitions from one stereo pair to another, and with the slightest change of position, the observer notices the glare and boundaries of each of the stereo pairs. Such phenomena can cause the observer a certain discomfort, which interferes with working with stereo images and an adequate perception of visual information on a stereo frame. The problems of visual discomfort in virtual and mixed reality systems are described in detail in [17].

In order to create the most comfortable stereo image for the viewer with the maximum stereo effect, during the computational experiments it was necessary to study the effect of various parameters on the quality of the stereo image.

During the experiments, when constructing the inscription, various parameters varied: font size, rotation angle on each frame in multi-view, and the distance between frames for a linear stereo image.

Due to the lack of a theoretical part in this area, it was necessary to search empirically with numerous tests for the necessary linear and angular shift parameters for a multi-view method. The optimal parameters were found, in which the transitions between stereo pairs became invisible to the viewer, and the inscription itself acquired volume and became optimal for the viewer to perceive in three-dimensional form.

At this stage of the work, when constructing, the combination of linear and angular displacement was first used. This made it possible to achieve a volume effect comparable to the effect achieved in a classic stereo setup.

It was found that, in principle, these parameters remain the same for very different font sizes. A similar effect was obtained for the inscription, where the font size was 32, and the most successful version of the inscription, where the font size was 66 (Fig. 1).

 

Fig. 1. The construction of stereo images of the inscription with a font size of 66.

 

In the case of constructing a depth map for the inscription, it was revealed that the desired object should have clearly expressed depth and volume. The algorithm for constructing a depth map, involving the alignment of pictures — rectification — and the search for the corresponding pairs of points is described in detail in [14–15] and implemented using a special software package [16]. For example, a text variant suitable for a multi-view representation (Fig. 1) turned out to be unsuitable for constructing a depth map due to insufficient volume of letters.

Figure 2 shows a depth map for an inscription with pronounced depth and volume.

 

Fig. 2. Construction a depth map for the whole inscription.

 

As a result of all experiments, a clearly visible stereo effect was achieved and the conditions necessary for constructing depth maps were identified.

However, in the process, the following problem was discovered - the presence of artifacts clearly visible on the autostereoscopic monitor. In the above image they are also visible, only on depth maps. This problem arises due to the lack of information for constructing an image for areas that are not visible in the original image, but are observed in a shifted angle.

The reasons are that the algorithms for smoothing the constructed depth map used in the software package used in the study work well mainly with real stereo photographs when the visualization object does not contain sharp edges with a contrasting color change. In our case, the object of visualization is text, most often possessing such properties. Thus, further studies suggest the selection and application of the most effective smoothing algorithms.

3. Methods of combining image and text information

After solving the problem of creating stereo labels on their own, as separate frames in a stereo presentation or stereo film, an equally important sub-task arises - combining the image and text information in one frame. Currently, in the case of demonstration of scientific results to observers, very many objects require accompanying information located directly on the same frame as the image. In many cases, the signature and the object cannot be separated into different frames, since they make up a single logical display of information. For example, when depicting a coordinate system, one cannot fail to mark a designation for each of the coordinates. An example is shown in Figure 3. This figure is not informative, like a graph, because it does not carry accurate data and is not bound to coordinates. This is a simple three-dimensional model, which is a schematic three-dimensional graph.

 

Fig. 3. Three-dimensional model of the coordinate system in volume.

 

The constructed nine-view image gave a stereo effect, which was recognized by observers as satisfactory. However, the presented figure does not carry an informative load, since it is not accompanied by the necessary additional information, and the viewer does not even have an idea about the coordinate system.

Figure 4 shows a more complete image. The coordinates are signed on it, and the letters indicating the coordinates also provide a stereo effect.

 

Fig. 4. Three-dimensional model of the coordinate system in volume with signed coordinates.

 

To construct this stereo image, an image matrix was used (Fig. 3), to which volume letters were added and sequentially shifted and rotated on each frame by the same interval experimentally calculated earlier. As a result, a stereo image with a volume signature was obtained, the stereo effect of which was pronounced. Along with this, an additional effect was revealed when the viewer from different positions saw that the letters are on different planes each time. For example, the location of the letter Z was perceived by observers either behind the horizontal axis or in front of it, depending on the viewer's location in a particular observation sector in front of the screen of the autostereoscopic monitor.

For an integral label, completely located on top of the image, such problems do not arise. Figure 5 shows the image of the simulation results of a supersonic flow around a cone at an angle of attack with the corresponding inscription.

 

Fig. 5. Image of simulation results of a supersonic flow around a cone with the corresponding signature.

 

As a result, the inscription was located on top of the cone, but behind its tip, which in turn was perceived by the audience as protruding from the screen by several tens of centimeters. As a result, another important parameter was revealed - the remoteness of the object from the plane of the screen and its correlation with other objects in the frame. Further experimental studies are planned to be directed at controlling the remoteness of an object from the plane of the screen and other objects depending on the stereo base.

4. Conclusion

This work is a continuation of a works series devoted to the implementation of the project to build stereo presentations of the results solving mathematical modeling problems. The results of numerical experiments on the presentation of textual information on an autostereoscopic monitor that allows the construction of stereo images using depth and multi-view maps are presented. The problem of combining images and textual information in one frame for a multi-view presentation is considered. As a result of practical experiments, the most clear and expressive stereo effect was achieved.

Further research is planned to be directed to the development of a practical technology for controlling the remoteness of an object from the plane of the screen and other objects depending on the stereo base and its correlation with other objects in the frame.

References

[1] Generation of Stereo-Presentations in Photorealistic Rendering and Scientific Visualization /Andreev S.V.[et al] // Keldysh Institute preprints, 2010. ¹ 61. 16 p. http://library.keldysh.ru/preprint.asp?id=2010-61[inRussian].

[2] Synthesis of photorealistic three-dimensional images in modern presentation systems / Andreev S.V.[et al] // Software&Systems  2007.  ¹ 3. p. 37-40. [in Russian].

[3] Andreev S., Filina A. Using stereo presentations for visualization of scientific calculations results / Scientific Visualization. 2012. v.4. ¹ 1. p.12-21.

[4] Hardware-software complex of 3D presentations based on a virtual studio and virtual environment / Vandanov V.G. [et al] // Proceedings of the 1-st international conference «3D visualization of scientific, technical and social reality. Cluster technologies of modeling» Izhevsk. 2009. P.73-77. [in Russian].

[5] Mezhenin A.V., Tozik V.G. 3D Visualization using the stereo image effect // Proceedings of the 2-nd international conference «3D visualization of scientific, technical and social reality. Cluster technologies of modeling» Izhevsk. 2010. [in Russian].

[6] Mikhaylyuk M.V., Huraskin I.A. Synthesis of stereo images for virtual reality systems using an optical tracking system / Software&Systems  2006.  ¹ 3. p. 10-14. [in Russian].

[7] Andreev S.V. [et al] Modelling and visualisation of blade assembly with complicated shape for power turbine / Scientific Visualization. 2015. v.7. ¹ 4. p.1-12

[8] Torgashev M.A., P.Y. Timokhin. The technology of stereo video files’ synthesis for the system of 3D real-time visualization / Software Products and Systems, 2012, ¹ 3, pp. 74-80. [In Russian]

[9] Torgashev M.A. Implementation of stereo mode for various devices for real time displaying / Software Products and Systems, 2010, ¹ 2, pp. 23-29. [In Russian]

[10] Mikhaylyuk M.V., Maltsev A.V. Timokhin P.Yu. The methods of 3D stereo visualizationof data obtainedin simulation of unstable oil displacement from porous media / Proceedings of Scientific Research Institute for System Analysis  RAS, 2018, v.8, N 2, p. 125-129. [in Russian].

[11] Visibility Experiment and Evaluation of 3D Character Representation on Mobile Displays / Hiromu Ishio [et al.]  //  C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp. 46–51, 2011.

[12] Andreev S.V., Bondareva N.A. Constructing a representation of textual information in stereo presentations // Proceedings of the 28-th International Conference of Computer Graphics and Vision GraphiCon-2018, Tomsk, Russia, 23-28 September 2018, p. 86-89. [in Russian].

[13] S.V.Andreev, A.E.Bondarev, V.A.Galaktionov, N.A.Bondareva (2018) The problems of stereo animations construction on modern stereo devices. Scientific Visualization 10.4: 40 - 52, DOI: 10.26583/sv.10.4.04

[14] S.V.  Andreev, N.A. Bondareva, E.Yu.  Denisov. Stereo Presentations Problems of Textual information on an Autostereoscopic Monitor (2019). Scientific Visualization 11.4: 90 - 101, DOI: 10.26583/sv.11.4.08

[15] Kalinichenko A.V., Sveshnikova N.V., Yurin D.V. Epipolar geometry and evaluation of its reliability from the results of the restoration of the three-dimensional scene by factorization algorithms // Proceedings of the 16-th International Conference of Computer Graphics and Vision GraphiCon-2006, 2006. [in Russian].

[16] 3D Stereoscopic Photography, http://3dstereophoto.blogspot.com/ last access February 23 2020.

[17]. A.D. Zhdanov, D.D. Zhdanov, N.N. Bogdanov, I.S. Potemin,V.A. Galaktionov, and M.I. Sorokin. Discomfort of Visual Perception in Virtual and Mixed Reality Systems   // Programming and Computer Software, 2019, Vol. 45, No. 4, pp. 147–155.  DOI:  10.1134/S036176881904011X