1. Introduction

Currently, space industry is one of the advanced spheres of human activity where virtual environment systems (VES) are highly demanded [1]. Real-time computer generation of realistic virtual analogue of space environment opens up new opportunities for training cosmonauts [2, 3], research and development of advanced space systems [4, 5], planning various missions [6, 7], etc.

An important part of plausible virtual space environment are dynamic shadows of terrain of planets and its satellites. A promising approach to obtain high-quality, geometrically accurate terrain shadows is ray casting, a special case of ray tracing widely applied in photorealistic rendering [8]. In the context of VES, terrain shadows obtained by ray casting significantly increase reliability, since source digital elevation model - height map is used -, and there is no dependency on 3D terrain model construction and visualization methods [9] which may contain inaccuracies, hidden vulnerabilities, as well as slow down rendering.

Shadow ray casting implementation based on classical voxel traversal algorithm [10] is often considered as reference, where every texel of height map, lying on ray trace is tested (for hard shadows - a ray per each pixel of terrain image). In practice, it can take a long time and even results in offline (in the case of detailed height maps), that is unacceptable for VES. An effective way is to reduce ray casting complexity and parallelize rays processing on the GPU (for instance, CUDA-based uniform sampling [11], cone map [12], maximum mipmaps [13, 14], etc.). However, because of high focusing on the quality and performance of shadow rendering, issues of integrating GPU-accelerated ray-casting of terrain shadows into VES remain often out of scope. As practice shows, it can seriously impact overall performance and extensibility of the VES, therefore effective technologies resolving this task are required. This paper describes a solution for our VES VirSim [2], developed at FSI SRISA RAS. Section 2 describes core technology of 3D terrain modeling and visualization, implemented in the subsystem of the VirSim complex. Section 3 discusses a technology of integrating ray casting of terrain shadows in visualization pipeline. Section 4 presents rendered terrain shadows and comparison with a real photo.

2. Modeling and visualization core technology

In our core technology, a 3D model of visible terrain, that meets the requirements of  simulation of orbital observations, is constructed and visualized in real-time [15]. Figure 1 shows main stages of the technology, consider them briefly.

 

Fig. 1. Modeling and visualization pipeline.

 

At the stage of dynamic properties simulation, dynamic parameters of the visible motion (at viewport) of terrain surface along daily orbit tracks of the spacecraft are calculated (taking into account self-rotating of the planet). Motion simulation is performed based on classical Kepler equation (for an elliptical orbit), but according to the modified scheme, where world coordinate system is placed in spacecraft model, and terrain model is located and rotated in such a way as to simulate a flight along orbit track [16]. The stage results in modelview and projection matrices defining visibility sector of terrain model.

In visibility sector, at the stage of geometric properties modeling, 3D model of the terrain relief is constructed in real-time. Relief construction is performed entirely on the GPU based on the original adaptive tessellation (triangulation) scheme with vertex displacement according to detailed height map [17].

For the 3D model constructed, at the stage of visual properties modeling, a number of mappings with various texturing (daytime underlying surface, night city lights, cloud cover, and etc.) with details close to viewport resolution, are created. The mappings are synthesized in textures of viewport size, called screen maps. All screen maps are synthesized on the GPU in real-time, based on extracting, swapping and rendering visible tiles (pages) of corresponding extra-large textures [18].

Using screen maps, at the stage of light-shadow modeling, the resulting image of terrain model with physically correct light-shadow appearance is produced. Light-shadow appearance is calculated on the GPU at real-time, and includes the following steps:

(1) calculating screen map of illumination (covering atmosphere, cloud cover and relief) in a high dynamic range, taking into account light absorption and scattering in the atmosphere (dawn / day / sunset lighting, twilight, terminator zone),

(2) calculating screen map of terrain shadows in a high dynamic range (dynamic change in sunlight direction is supported),

(3) tone mapping synthesized light-shadow appearance into a standard range of visualization device (for correct color and brightness transmit of illumination modelled).

The description of the first and third steps can be found in our previous researches [19] and [20], so let's take a closer look at the technology of implementing the second step.

3. The technology to produce screen map of terrain shadows

In VES, it is quite common for a single visualization session to synthesize images of virtual terrain model for observers with different fields of view and viewports. If viewport sizes are small enough, then it is irrational to spend computational resources (and time) on constructing terrain shadows based on detailed source height map (used to produce adaptive relief model, see Section 2), because details of the shadows will not be noticeable. Instead, in this paper, it is proposed to use a mip-pyramid [21] of detailed source height map and choosing an acceptable level of detail (LOD) for constructing shadows (see Figure 2).

 

Fig. 2. Production diagram of screen map of terrain shadows.

 

To implement this idea, we expanded the technology obtained earlier [22] to search for shadow rays on the GPU using accelerated ray casting. The expanded technology includes the following two stages.

At the first stage, a mipmap of local height extrema from height map of the 0th (most detailed) LOD is created. In fact, it is dual channel (R, G) mip-texture, where distances and from sea level surface to upper and lower local equipotential surfaces are stored (see Figure 3). This mipmap is created at preprocessing stage and loaded into video memory before the visualization (yellow path in Figure 2). Figure 4 shows an example of created local extrema mipmap (first four mip-levels).

 

Fig. 3. Local height extrema.

 

Fig. 4. Example of local extrema map ({r, g, b} = {h_max, h_min, 0}).

 

At the second stage, a screen map of terrain shadows is constructed, using a height map of a certain LOD chosen from the mip-pyramid and the mipmap of local height extrema. Let's consider the principle of constructing screen shadow map using the example of the task of detailed observation (photographing) certain terrain areas from the space (narrow field of view, about 5 degrees).

Let's create a single-channel texture of floating point format and of size (in pixels) same as viewport, and run parallel shader threads on the GPU, in which texel values of the texture will be calculated. A standard approach to implement it is to create a rectangle polygonal model of viewport size and render this rectangle into the texture using the developed fragment shader. To understand which shader threads will process texels related to terrain surface (and may contain shadows), it is convenient and effective to use screen map of texture coordinates of terrain surface (ST-map), which is synthesized at the stage of modeling visual properties in our core technology (see Section 2). ST-map is a dual-channel (R, G) texture of viewport size, where each texel keeps texture coordinates ( , ) of visible point of terrain surface model or code value indicating its absence. Figure 5 shows an example of synthesized ST-map.

 

Fig. 5. Screen map of texture coordinates (color channel B is set to 0).

 

Having read texture coordinates from the ST map, fragment shader thread starts ray casting loop, accelerated by means of mipmap of local height extrema. In this loop, a traverse from texel to texel of height map of LOD along the trace of inversed sun ray is performed. During traversing the texel groups (according to mipmap of local extrema) whose maximum heights are less than sun altitude, i.e. don't occlude passing sun ray, are skipped (see Figure 6).

 

Fig. 6. Traversing the height map.

 

Loop termination is possible in three cases: a) if the end of active area of height map is reached (went beyond the horizon), b) if sun ray is occluded by minimum height of any texel group, c) if sun ray is occluded by the height of LOD texel. In the first case, 1 (no shadow) is written into the texel of screen shadow map, and in the last two cases, the shadow coefficient :

,

where , correspond to the darkest and the brightest shadows, is unit vector directed to the sun and is the normal at the corresponded point of sea level surface (specified by ( , ) coordinates from ST-map). Figure 7a shows synthesized screen map of terrain shadows, Figure 7b – screen map of daytime underlying surface, and Figure 7c – the result of multiplying two previous screen maps.

 

Fig. 7. Synthesizing the light-shadow appearance of terrain model: (a) screen map of terrain shadows; (b) screen map of daytime terrain without shadows; (c) resulting image of terrain model with shadows.

4. Results

Based on the proposed technology, dynamic terrain shadows were implemented in the subsystem of the VirSim complex. To test the developed solution, a photo of a real terrain area with pronounced shadows was found, a virtual model of this area (with similar location of light source and camera) was created, and visualization frame of the model with the original photo was compared. As the reference, a photo of the Andes [23] taken by Russian cosmonaut Fyodor Yurchikhin from the International Space Station (ISS) was used (see Figure 8, top). Unfortunately, exact geo-references of the depicted objects were not pointed out, so it took some research (based on characteristic shape of the lake) to find out that the photo shows Lake Chungara and the Parinacota volcano ( , , Chile). Using these coordinates, the corresponding terrain area on the ALOS World 3D - 30m global height map [24, 25] was found and its local height map of size was prepared. Based on the obtained height map, terrain shadows visualization was performed (see Figure 8, bottom). Also a sequence of performance tests at different sun altitudes for LOD 0 and LOD 1 of Parinacota height map were performed (see Table 1).

 

Fig. 8. Shadows of the Parinacota volcano (Andes, Chile): (top) real photo from the ISS (source: © State Space Corporation ROSCOSMOS, author Fyodor Yurchikhin [23]); (bottom) a frame of shaded terrain model visualization (source data from ALOS World 3D - 30m [24, 25]).

 

Table. 1. Performance (in frames per second) of ray-casting terrain shadows implementation.

LOD	Resolution/ Sun altitude
0		90	89	86	67	37
1		114	113	110	82	48

 

All tests were done at 1920x1080 resolution on the PC (Intel Core i7-6800K 3.40GHz, 16Gb RAM, NVidia GeForce RTX 2080, 16x AF, 8x AA).

A comparison of the images in Figure 8 shows that the developed solution reproduces shapes and location of terrain shadows well, even in conditions of limited resolution of the local height map (about 30 m/pixel at the equator is the highest resolution of global height maps currently available in the public domain).

5. Conclusions

In this paper, an original technology to integrate dynamic terrain shadows into a virtual environment system is considered. The technology benefits in geometrically accurate shadows provided by shadow ray-casting implementation and minimally invasive integration mechanism based on production and application of screen map of terrain shadows, easily integrated into visualization pipeline. The form of screen map also allows some "weather" correction of modeled shadows (for example, blurring contours for cloudy weather) to be performed in real-time. Thanks to high self-sufficiency, the entire shadow modelling process can be effectively taken out into a separate computational thread which can be executed in parallel on the GPU using special high-performance ray tracing cores (RTX architecture). It can preserve valuable resource of uniform computing GPU-cores necessary for operating of main visualization system.

6. Acknowledgements

The publication is made within the state task of Federal State Institution “Scientific Research Institute for System Analysis of the Russian Academy of Sciences” on “Carrying out basic scientific researches (47 GP)” on topic No. FNEF-2022-0012 “Virtual environment systems: technologies, methods and algorithms of mathematical modeling and visualization. 0580-2022-0012”.

References

1. Altunin A.A., Dolgov P.P., Zhamaletdinov N.R., Irodov E.Yu., Korennoy V.S. Application of Virtual Reality Technologies in Training Cosmonauts for Extravehicular Activities // Scientific Journal Manned Spaceflight. – 2021. – No 1(38). – pp. 72–88. [in Russian] (doi: 10.34131/MSF.21.1.72-88) (http://www.gctc.ru/media/files/Periodicheskie_izdaniya/ppk_2021_1_total_38/5_stat.a_altunin_dolgov.pdf)

2. Mikhaylyuk M.V., Maltsev A.V., Timokhin P.Y., Strashnov E.V., Kryuchkov B.I., Usov V.M. The VirSim Virtual Environment System for the Simulation Complexes of Cosmonaut Training // Scientific Journal Manned Spaceflight. – 2020. – No 4(37). – pp. 72–95. [in Russian] (doi: 10.34131/MSF.20.4.72-95) (http://www.gctc.ru/media/files/Periodicheskie_izdaniya/ppk_2020_4_total_37/5_stat.a_mihailuk_pq.pdf)

3. Space Center Houston. How NASA uses virtual reality to train astronauts, 2018 (https://spacecenter.org/how-nasa-uses-virtual-reality-to-train-astronauts/).

4. Strashnov E.V., Finagin L.A., Torgashev M.A. Motion Control Simulation of Virtual Jet Pack Model for Cosmonaut Self-Rescue // Informacionnye tekhnologii i vichslitel’nye sistemy [Journal of Information Technologies and Computing Systems]. – 2021. – No 2. – pp. 94–104. [in Russian] (doi: 10.14357/20718632210210).

5. Lan C., Xu Q., Li J., Zhou Y. A Virtual Space Environment Simulation System // Virtual Reality, Second International Conference, ICVR 2007. Lecture Notes in Computer Science, Springer, Vol. 4563, 2007, pp. 497–503 (doi: 10.1007/978-3-540-73335-5_54).

6. Piovano L., Basso V., Rocci L., Pasquinelli M., Bar C., Marello M., Vizzi C., Lucenteforte M., Brunello M., Racca F., Rabaioli M., Menduni E., Cencetti M. Virtual Simulation of Hostile Environments for Space Industry: From Space Missions to Territory Monitoring // Virtual Reality - Human Computer Interaction, IntechOpen, 2012, pp. 153–178 (doi: 10.5772/51121) (https://www.intechopen.com/chapters/38748).

7. Greenstein Z. For Astronauts, Next Steps on Journey to Space Will Be Virtual, 2016 (https://blogs.nvidia.com/blog/2016/08/01/astronauts-next-steps-journey-space-will-virtual/).

8. Frolov V.A., Voloboy A.G., Ershov S.V., Galaktionov V.A. Light Transport in Realistic Rendering: State-of-the-Art Simulation Methods // Programming and Computer Software. – 2021. – Vol. 47, No 4. – pp. 298–326 (doi:10.1134/S0361768821040034).

9. Brawley Z., Tatarchuk N. Parallax Occlusion Mapping: Self-Shadowing, Perspective-Correct Bump Mapping Using Reverse Height Map Tracing // ShaderX3: Advanced Rendering with DirectX and OpenGL, 1st. ed., Charles River Media. – 2004. – pp. 135–154.

10. Amanatides J., Woo A. A Fast Voxel Traversal Algorithm for Ray Tracing // Proceedings of the 8th European Computer Graphics Conference and Exhibition (Eurographics ’87), Amsterdam. – 1987. – pp. 3–10 (https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.42.3443).

11. Aslandere T., Flatken M., Gerndt A. A Real-Time Physically Based Algorithm for Hard Shadows on Dynamic Height-Fields // Proceedings of 12. Workshop der GI-Fachgruppe on Virtuelle und Erweiterte Realität, Aachen Verlag, Bonn. – 2015. – pp. 101–112   (https://elib.dlr.de/101497/).

12. Policarpo F., Oliveira M.M. Relaxed Cone Stepping for Relief Mapping // in: H. Nguyen (Ed.), GPU Gems 3, Addison-Wesley Professional. – 2007. – pp. 409–428 (https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping).

13. Tevs A., Ihrke I., Seidel H.-P. Maximum Mipmaps for Fast, Accurate, and Scalable Dynamic Height Field Rendering // Proceedings of the 2008 Symposium on Interactive 3D Graphics and Games (I3D ’08), New York. – 2008. – pp. 183–190 (doi:10.1145/1342250.1342279).

14. Jung D., Schrempp F., Son S. Optimally Fast Soft Shadows on Curved Terrain with Dynamic Programming and Maximum Mipmaps, 2020 (https://arxiv.org/pdf/2005.06671.pdf).

15. Mitin A.I., Bragin V.I. Ways of Improving the Adequacy of Modeling Visual Conditions of the Earth Surface Monitoring on the Service Module Simulator of the ISS RS // Scientific Journal Manned Spaceflight. – 2014. – No 3(12). – pp. 60–70. [in Russian] (http://www.gctc.ru/media/files/Periodicheskie_izdaniya/ppk_2014_3_total_12.pdf).

16. Mikhaylyuk Ì.V., Timokhin P.Yu. Simulation of mutual planet-satellite arrangement in space video simulators, Mathematica Montisnigri XXIX. – 2014. – pp. 91–97 [in Russian] (https://lppm3.ru/files/journal/XXIX/MathMontXXIX-Mikhaylyuk.pdf).

17. Timokhin P.Yu., Mikhaylyuk M.V., Maltsev A.V. Real-time construction on the GPU of adaptive terrestrial relief model based on the spheroid // International Journal of Open Information Technologies. – 2019. – Vol. 7, No 10. – pp. 22–35 [in Russian] (http://injoit.ru/index.php/j1/article/view/825/788).

18. Timokhin P.Yu. Textured planet model visualization system for space experiments simulation // Programmnye produkty i sistemy [Software & Systems]. – 2015. – No 4. – pp. 99–104 [in Russian] (doi: 10.15827/0236-235X.112.099-104) (http://www.swsys.ru/index.php?id=4076&page=article).

19. Timokhin P.Yu. Real-time virtual Earth illumination simulation given the atmosphere // Trudy NIISI RAN [Proceedings of SRISA RAS]. – 2014. – Vol. 4, No 2. – pp.  78–84 [in Russian] (https://www.niisi.ru/tr/2014_T4_N2.pdf).

20. Mikhaylyuk M.V., Timokhin P.Y., Torgashev M.A. The Method of Real-Time Implementation of Tone Mapping and Bloom Effect // Programming and Computer Software. – 2015. – Vol. 41, No 5. – pp. 289–294 (doi:10.1134/S0361768815050084).

21. Lengyel E. Mathematics for 3D Game Programming and Computer Graphics, 3rd. ed., Course Technology, Boston, MA, 2012.

22. Timokhin P.Yu., Mikhaylyuk M.V. Reliable GPU-based methods and algorithms of implementation dynamic relief shadows in virtual environment systems // Proceedings of the 31th International Conference on Computer Graphics and Machine Vision (GraphiCon 2021). – 2021. – pp. 83–94 (doi: 10.20948/graphicon-2021-3027-83-94) (http://ceur-ws.org/Vol-3027/paper8.pdf).

23. State Space Corporation ROSCOSMOS. Yurchikhin Fyodor: The Earth is our homeland (2), The Andes (http://en.roscosmos.ru/media/gallery/big/20609/5166609507.jpg).

24. Takaku J., Tadono T., Doutsu M., Ohgushi F., Kai H. Updates of ‘AW3D30’ ALOS Global Digital Surface Model with Other Open Access Datasets // The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, ISPRS. – 2020. –Vol. XLIII-B4-2020. – pp.183–189 (doi:10.5194/isprs-archives-XLIII-B4-2020-183-2020).

25. JAXA, ALOS World 3D - 30m, (www.eorc.jaxa.jp/ALOS/en/aw3d30/data/index.htm).