1. Introduction

Virtual and augmented reality (VR/AR) technologies have already become one of the most important digital technologies in 2017, which served as the foundation for the implementation of the Russian Federation Digital Economy program. By this time, millions of foreign and domestic corporations have already become convinced that VR / AR reality is a convenient and effective tool that is necessary to solve various tasks while implementing digital transformation of production. Every year the technology presents more and more innovative projects to the production, as well as new opportunities that open up while researching the effectiveness of digital solutions [1]. Digitalization of society is now the main task of the economy, which in turn is impossible without digitalization of production.

VR/ AR tools can be actively used throughout the entire product lifecycle: to identify errors at the earliest stages of design before passing mock-up commissions, to improve ergonomics and the entire production process as a whole, simulating the processes of operation, modernization and repair.

VR /AR technologies allow you to be at the operator's workplace inside the designed product, check its various operational characteristics, technical requirements, access to various components of the product for their convenient installation and repair.

The scope of application of VR / AR technologies in production is actively developing to simulate the installation of new equipment, which saves time spent on its design. The developed equipment models can be used at the same time to train operators working on it. As a result, employees at the new site will be immediately ready to proceed to work, saving time on training.

Such parallel processes make it possible to speed up the setup of production lines and equipment, which will have an economic effect for the enterprise [2]. Virtual and augmented reality technologies allow for more efficient equipment layout, allow you to choose the optimal color solutions for the designed products that meet the customer’s needs. With the help of VR / AR technologies, it is possible to simulate an emergency situation. It is often quite dangerous and expensive to experience unforeseen scenarios in reality, so the staff is only familiar with the theory of how to act in case of emergency. Virtual reality allows the company's employees to be more prepared how to act in any dangerous situation. The digital space allows you to design and discuss the results, simultaneously combining specialists of various fields in a single information field, who do not need to be in the same room, but are required to be in the same virtual space, which is provided by VR helmets and special software. All this allows you to save time on business trips and coordination of projects.

Virtual reality systems have been used by giant companies for quite a long time. For example, Boeing, Lockheed Martin used the technology "VR-cave" ("caves" of virtual reality). For quite a long time, "VR caves" and projection stereo screens (CAD-Wall) have been the main VR system for the implementation of industrial tasks. These systems allowed development teams to be in the same virtual space, while only one person could interact with VR [1].

Currently, many projects with virtual and augmented reality are being implemented in the world, for example, in the metalworking industry, in the equipment monitoring system and in analytical services that help personnel in equipment diagnostics, fault prediction, etc. The implementation of the technology in production is described in [3, 4]. It has significant advantages over analytical services visualization on computer screens. The virtual world contains software consisting of services, modules and algorithms and can control the physical world. The monitoring system uses big data collected from a variety of sensors installed on the equipment. For their analysis, it is necessary to process the received data for subsequent human analysis (see Fig.1).

The 3D Vista Virtual Tour Pro software environment allows you to create virtual tours in virtual space, allowing, among other things, to conduct personal and group interactive excursions with a guide [5]. In 2020, the developers of this software environment significantly improved its functions designed for e-learning, including the creation of quests that allow you to consolidate your knowledge, as well as added automatic integration with LMS Moodle [6]. These functions are the basis for interactive learning.

 

Fig. 1. Diagram of human interaction with an equipment digital prototype

 

With this software, you can create high-quality panoramas using, among other things, 360° technology (photo and video shooting in 360 degree format) and support for modern virtual reality devices: Samsung GearVR, Oculus, HTC Vive, Google Cardboard.

One of the advantages of the program is that virtual tours created using 3D Vista Virtual Tour Pro software are multiplatform and can be viewed on Mac, PC, iPhone/iPad and Android in any modern Web browser (Firefox, Microsoft Edge, Chrome, Safari, etc.) without downloading any additional programs, players, or plug-ins.

Previously the authors carried out work related to the virtual reality development and application in the field of historical objects reconstruction [7, 8], as well as in the field of education [9, 10].

Currently, many industrial companies are implementing virtual production concepts to counter global competition and major production problems. Under these conditions, the use of virtual production and its digital representation makes these processes even more important for production optimization. In the process of rapidly changing information, technologies, digital tools and systems are used in all industries to manage a variety of tasks throughout the product lifecycle [11].

In this regard, this paper examines the issues of digital transformation of industrial production into the educational process, which ensures the education quality improvement, exchange of opinions, mutual consultation.

2. Creation of an industrial production digital prototype

When implementing the technology of creating an industrial production digital prototype on the example of a machine-building plant, at the first stage, a layout of the entire territory is created, including the adjacent territories with the natural ecosystem around it, using the Twinmotion program, which provides architectural 3D visualization [12, 13].

Twinmotion is a kind of "wrapper" over the Unreal Engine, designed specifically for creating photorealistic architectural and landscape visualizations [14]. The user is given the opportunity to change the materials of the elements of the scene, vary the season and weather, set the time of day, shape the landscape, add artificial lighting and environmental sounds on the location. It is possible to import three-dimensional models in various graphic formats, and at the same time there are no problems with the number of polygons. The system allows you to place and configure virtual cameras in the required places of the location, to obtain high-quality conventional images and videos, and video and panoramas in 360° format. This software product has an advanced lighting and shadow system. In particular, a physical model of the atmosphere, sun and sky is used, accurately reflecting the real geographical location of the scene, the seasons and time of day. Effects such as fog, haze, dust or smoke are supported as well. The vegetation system is represented by highly polygonal models of trees, bushes and grass from the Xfrog Xlang and Megascans collections, and different models are used for each season. Such a tool as Twinmotion Presenter should also be mentioned, which allows you to prepare a project as a separate executable file with all the necessary resources for offline viewing. Both the Twinmotion system itself and the Twinmotion Presenter support virtual reality glasses, in particular Oculus Rift, HTC VIVE.

In the Twinmotion system, a 3D scene is created, including terrain, roads, utility infrastructure systems, as well as imported three-dimensional models of industrial facilities made in 3D modeling programs (SketchUp, Blender, Archicad, etc.) [8]. Thus, a model of the entire production is created in Twinmotion (see Figure 2).

 

Fig. 2. Visualization of the general view of a machine-building plant digital model

 

Next, you need to place and configure virtual cameras in the required places of the scene, and then get high-quality videos and images rendered by Twinmotion, including videos and images in 360° format.

At the next stage, an interactive virtual tour is created using the 3D Vista Virtual Tour Pro software environment, which uses multimedia materials obtained at the previous stage. The created tours can be viewed online and offline on any device and without the need to install any special software or plug-ins (see Figure 3). A virtual tour of the machine-building enterprise is located at: https://heritage.tstu.ru/memorial/directaccess/zavod/index.htm .

The created virtual tour in the 3DVista Virtual Tour Pro environment is used for a variety of tasks: online discussion of the designed equipment 3D model; staff training; emergency situations drills; completing onsite educational quests. As an example, the problems of equipment layout and pipeline systems tracing in the production room, as well as the implementation of technological processes for the manufacture of individual parts and assemblies of machine-building products were considered [15-18]. In particular, in the virtual space it is possible to analyze compliance with restrictions on: mutual arrangement of machine tools; non-intersection of machine tool service areas and building structure elements; dimensions of passages; distance between pipelines; distance between pipelines and machines, etc. [19-23].

 

Fig. 3. Visualization of one of the virtual tour panorama fragments inside a plant workshop

 

When implementing technological processes for manufacturing machine-building products, it is possible to visually verify the availability of: the necessary types of materials (steels) used for the manufacture of parts (sheet, strip, circle, square, etc.) in the warehouse; the main technological and auxiliary equipment, devices, auxiliary materials, etc. [24-26].

 

Fig. 4. Visualization of one of the panorama fragments inside one of the plant's workshops during an online discussion on the equipment functioning

 

A panorama fragment inside the plant shop visualization during an online discussion on the equipment functioning is shown in Fig. 4, which shows the control panel of the robotic complex on the left, and the KUKA KR 6 robot in the foreground. With the help of this complex, frame elements of various configurations welding is carried out. Robotic welding is a fully automated welding process implemented with the help of robotic manipulators and special welding equipment.

Figure 5 illustrates one of the production sites, which includes the KUKA robotic welding complex. The first robot (on the left) takes a steel frame from a special pallet and places it on the main conveyor. Then the frame moves to the second robot, which takes the steel sheet for bending from a special pallet and puts it on the prepared frame. The assembly (frame + steel sheet) moves to the next robot. The third robot welds the assembly together and then the welded structure moves to the last robot and is placed on a pallet.

To implement this function, the Live Guide Tours tool is used, which is included in the 3D Vista Virtual Tour Pro package and allows you to conduct video conferences inside the virtual tour in real time (see Figure 4). Thus, the teacher can conduct lectures, practical and credit classes remotely. Figure 5 illustrates a stage of the quest, during which the acquired knowledge is tested.

The acquired knowledge can be tested in the framework of e-learning using the LMS Moodle system by completing quests [27-30]. As an example, Table 1 shows a fragment of the questions database for testing certain machine tools types operating skills and knowledge (correct answers are marked in bold).

 

Fig. 5. Visualization of the production site, including the KUKA robotic welding complex

 

The Live Guide Tours system does not support special virtual reality devices such as glasses and helmets and functions on computers, laptops, tablets and mobile devices.

 

Table 1. A fragment of the questions database for testing certain machine tools types operating skills and knowledge

Questions	Response options	The number of points for the correct answer	Maximum response time (min.), ∞ - no restrictions
How many degrees of mobility does the KUKA KR 6 and KUKA KR 16 robot have?	3	10	5
	4
	6
In which coordinate system do the KUKA KR-6 and KUKA KR-16 robots work?	Cylindrical	10	5
	Angular
	Cartesian
What is the service coefficient of the KUKA KR 6 and KUKA KR 16 robots?	Power on - start assembly -start welding -start cycle	10	∞
	Power on - start cycle - start assembly - start welding
	Start cycle - start assembly - start welding - power on
…	…	…	…

 

To provide participants with the opportunity to meet and interact through avatars inside virtual reality, the authors used a multi-user social VR application Bigscreen [31, 32]. A feature of the Bigscreen system is that users can not only communicate in one virtual location and, for example, watch movies together in a virtual cinema hall, but also show their computer screen to other users connected to the system. This gives a unique opportunity to broadcast the window of any application running on the computer to other participants on a large virtual screen displayed inside virtual reality devices, thus organizing the process of collaboration, learning, etc. Using virtual environment allows you to dive deeper into the topic under discussion, visualize ideas using two-dimensional and three-dimensional content. The advantages of such virtual communication include a closer connection with the audience due to the effect of presence: participants are much less distracted by real-world events that occur during training. While wearing virtual reality glasses, it is impossible to check the phone, write e-mails or engage in any other activity unrelated to the current meeting. Virtual space visualization is available to all Internet users and is implemented using virtual reality glasses: Oculus Quest, Oculus Go, Oculus Rift, HTC Vive, and Windows MR. Fig. 6 illustrates a fragment of a visit to the machine-building plant production site by a network participant using the Bigscreen VR application, during which the sequence of switching on the control panel of a robotic complex was studied.

 

Fig. 6. Visualization of an equipment model fragment inside one of the enterprise workshops, presented in the Bigscreen VR application environment

 

Along with the use of VR technology, the authors suggest using AR technology when studying the technological processes of obtaining target products, for example, by supplementing the production line with missing equipment. As an illustration of AR technology, the readers of the magazine are invited to obtain the ABENE VHF-680 milling machine image from this page. To do this, you need to scan the QR code on Figure 7 with your smartphone, then open the web-page using Chrome or Safari browsers, allow access to the phone's video camera and point the camera at the marker. The image of the machine will appear on the phone screen.

 

Fig. 7. QR code (a) and marker (b) to obtain the image of the machine using augmented reality technology

 

3. Evaluation of the thematic educational virtual space effectiveness

To assess the created virtual space effectiveness of industrial production, the authors conducted a survey of various groups of users (students, professionals, teachers). Analysis in the coordinates "Importance-Performance" (Importance-Performance Analysis – IPA) is widely used to identify important characteristics with low performance rates [33].

As a part of the survey, users were asked to evaluate various characteristics (attributes) of the information educational resource on a five–point scale according to two criteria - how important each characteristic is for respondents (importance) and how well it is implemented (performance). The survey of students was conducted using Google Forms - a free online tool that allows you to create forms for data collection, online testing and voting [34, 35]. A fragment of the questionnaire form [36] is shown in Fig. 8. A total of 35 people was surveyed, who, it should be noted, had no previous experience in using virtual reality.

 

Fig. 8. Fragment of the questionnaire form for assessing the educational resource quality

 

After conducting a questionnaire for each of the characteristics, an average score was calculated for the importance and performance categories. The list of characteristics, as well as the average values of importance and performance, are presented in Table 2.

The obtained average values are then converted into a graph in which the Y-axis reflects the importance of the characteristic, and the X-axis reflects performance with the implementation of the characteristic (see Figure 9). The graph is divided into 4 quadrants (quarters): quadrant 1 (high importance, high performance); quadrant 2 (high importance, low performance); quadrant 3 (low importance, low performance); quadrant 4 (low importance, high performance). Fig. 10(a, b) represents histograms of characteristics (3) and (11) frequency ratings: the height of the columns indicates the number of corresponding ratings.

The graph shows that the bulk of the characteristics fell into quadrant 1 (high importance, high performance). This means that these characteristics fully satisfy the respondents' requests.

Characteristics (3) and (4) fell into the second quadrant (high importance, low performance). This means that this characteristic needs to be improved as soon as possible.

 

Table 2 List of the educational resource characteristics

Characteristic (attribute)	Question number	Importance	Performance
1. The scale of the virtual space of industrial production is sufficient to hold student's attention and motivate him/her to complete the study to the end.	1	4.29	4.26
2. How photorealistic is the simulated virtual space of industrial production.	2	4.17	4.46
3. The interesting content of the tour motivates the student to fully explore the industrial production in the virtual space.	3	4.26	3.97
4. The virtual space is easily navigable. The student can easily find the arrow-pointers along all the routes in the virtual space of the industrial production.	4	4.11	4.00
5. E-learning elements are presented in the form of quests in the industrial production virtual space.	5	4.16	4.17
6. Knowledge is obtained in an accessible game form.	6	4.03	4.14
7. Multimedia content and interactive tools are used in the virtual tour.	7	4.14	4.11
8. Integration with Learning Management Systems (LMS) is provided.	8	3.86	3.86
9. The information inside the industrial production virtual space is easily accessed and perceived.	9	4.09	4.13
10. Group work in the industrial production virtual space is possible.	10	3.94	4.29
11. A live guide and other participants can connect to the tour. Conducting an online tour with a live guide through the virtual space of industrial production is possible.	11	4.17	4.37
12. Immersion into the virtual space of industrial production using VR glasses or a VR helmet (Oculus, Vive, Gear VR equipment, etc.) is possible.	12	4.26	4.26
13. It is easy to navigate using virtual reality glasses.	13	3.97	4.06
14. Multiplatform implementation (Windows, Android, iOS, *nix) of the industrial production virtual space is supported.	14	4.16	4.14

 

Two characteristics fell into quadrant 3 (low importance, low performance): (8) and (13). It is likely that respondents are unclear why integration into the learning management system is required.

Characteristics (6) and (10) fell into quadrant 4 (low importance, high performance). This means that the developers of the industrial production virtual space devote too much effort to working on these characteristics.

 

Fig. 9. Importance – Performance Diagram

 

a)

b)

Fig. 10 (a, b). Histograms of characteristics 3 and 11 frequency ratings

 

Based on the results of the analysis in the coordinates "Importance-Performance", the following conclusions can be drawn. In general, all respondents indicate high importance and high performance with the implementation (the average values of importance are 4.08 and performance is 4.11). Considering the fact that the respondents had no previous experience in virtual reality, the WOW effect prevails, which forms a positive emotional attitude.

It is also worth noting that for an unprepared user, the first immersion in virtual reality is a distraction, new emotions and a new experience of exploring the virtual world distract from the subject of the study, as shown by characteristic (3).

The lack of experience in using special controllers at the beginning of work causes certain difficulties, as indicated by characteristic (4). However, these difficulties disappear quickly as practical experience is gained.

Thus, for educational purposes, virtual immersion is better used for an audience that has previously had virtual reality experience, or at least has completed an introductory course for beginners, which is provided by manufacturers of virtual reality devices.

Along with the industrial production virtual space quality assessment, the authors conducted a study using the LMS Moodle system to test the professional knowledge of students in the discipline "Design and Management of machine-building production". The results of testing of two groups of students with 12 people in each group, conducted with immersion in the virtual space of industrial production (group 1) and without it (group 2), showed for the first group a 12.5% higher proportion of correct answers to the questions of the tests. This is another confirmation that using an immersive educational environment is a reasonable way to improve the effectiveness of learning

4. Conclusion

As a result of the conducted research, the authors created a virtual educational thematic space dedicated to an enterprise which produces machine-building products, using modern software products such as Twinmotion, 3D Vista Virtual Tour Pro, Bigscreen, etc.

A digital prototype of a machine-building plant was created in the Twinmotion environment, providing immersive architectural 3D visualization.

The created virtual tour in the 3DVista Virtual Tour Pro environment is used for a variety of tasks: online discussion of the designed equipment 3D model; staff training; emergency situations drills; completing onsite educational quests.

The Live Guided Tours tool, included in the 3D Vista Virtual Tour Pro package, allows you to conduct video conferences inside a virtual tour. The guide, which can be a teacher, a colleague, can point out certain elements of the tour to other guests, discuss together what everyone sees in real time.

The use of a multi-user social VR application for virtual reality Bigscreen provides network participants with the opportunity to meet and interact through avatars inside rooms in virtual reality using special glasses and helmets.

Thus, the authors propose a technology for creating a digital prototype of industrial production, which can be used as a tool at the design and management stages of its functioning, as well as its application in the educational process while training highly qualified technical specialists - during lectures, practical and assessment classes with students, which forms the basis of an approach to the digital transformation of industrial production into the educational process. It allows to provide conditions for improving the quality of education, exchange of opinions, mutual consultation, which is confirmed by the results of evaluating the effectiveness of the created industrial production virtual space and of the students’ professional knowledge assessment.

References

1. Lämkull D., Zdrodowski M. The need for faster and more consistent digital human modeling software tools // Adv. Tran. Discip. Eng. 2020. Vol. 11. P. 299–310. DOI: 10.3233/ATDE200037

2. Choi S., Jung K., Noh S. Do. Virtual reality applications in manufacturing industries: Past research, present findings, and future directions // Concurr. Eng. 2015. Vol. 23, No. 1. P. 40–63. DOI: 10.1177/1063293X14568814

3. Vlah D., Čok V., Urbas U. VR as a 3D modelling tool in engineering design applications // Appl. Sci. 2021. Vol. 11, No. 16. P. 7570. DOI: 10.3390/app11167570

4. Greiner P. et al. Remote-audit and VR support in precision and mechanical engineering // Photonics and Education in Measurement Science 2019 / ed. Zagar B. et al. SPIE, 2019. P. 48. DOI: 10.1117/12.2533016

5. 3DVista. 3DVista – Virtual Tours, 360 ^o video and VR software [Electronic resource]. 2022. URL: https://www.3dvista.com/.

6. 3DVista. Virtual Tours in E-Learning, Training & Quizzing [Electronic resource]. 2020. URL: https://www.3dvista.com/en/blog/virtual-tours-in-e-learning-training-quizzing-v2/.

7. Nemtinov V.A., Gorelov A.A., Nemtinova, Yu.V., Borisenko A.B. Visualization of a virtual space and time model of an urban development territory // Scientific Visualization. 2016. Vol. 8, No. 1. P. 120–132. (In Russ.; abstract in Eng.)

8. Nemtinov V.A., Borisenko A.B., Nemtinova Y.V., Gorelov A.A., Tryufilkin S.V. Implementation of technology for creating virtual spatial-temporal models of urban development history // Scientific Visualization 2018. Vol. 10, No. 3. P. 99–107. DOI: 10.26583/sv.10.3.07

9. Nemtinov V. A., Borisenko A. B., Morozov V. V., Nemtinova Yu. V. Increasing the level of professional competence using a virtual educational environment // Vysshee Obrazovanie v Rossii = Higher Education in Russia. 2021. Vol. 30, No. 3. P. 104–113. DOI: 10.31992/0869-3617-2021-30-3-104-113 (In Russ.; abstract in Eng.)

10. Nemtinov V., Egorov S., Borisenko A., Morozov V., Nemtinova Yu. Support of design decision-making process using virtual modeling technology // Lecture Notes in Networks and Systems book series (LNNS,volume 432). 2022. P. 70–77. DOI: 10.1007/978-3-030-97730-6_7

11. Duda J., Oleszek S. Concept of PLM application integration with VR and AR techniques. 2020. P. 91–99. DOI: 10.1007/978-3-030-57997-5_11

12. Nemtinov V., Borisenko A., Gorelov A., Nemtinova Yu., Tryufilkin S. Development of a virtual library of cultural heritage objects for tambov region’s memorable places // International Multidisciplinary Scientific GeoConference Surveying Geology and Mining Ecology Management, SGEM. 2019. Vol. 19, No. 2.1. P. 225–231. DOI: 10.5593/sgem2019/2.1/s07.030

13. Wu L. et al. The design of virtual town based on twinmotion // 2020 International Conference on Virtual Reality and Visualization (ICVRV). IEEE, 2020. P. 348–349. DOI: 10.1109/ICVRV51359.2020.00096

14. Epic Games. A Cutting Edge Real-Time Architectural Visualization Tool - Twinmotion [Electronic resource]. 2022. URL: https://www.twinmotion.com/.

15. Gallegos C. et al. The use of a game-based learning platform to engage nursing students: A descriptive, qualitative study // Nurse Educ. Pract. 2017. Vol. 27. P. 101–106.

16. Lam J.T. et al. Use of virtual games for interactive learning in a pharmacy curriculum // Curr. Pharm. Teach. Learn. 2019. Vol. 11, No. 1. P. 51–57. DOI: 10.1016/j.cptl.2018.09.012

17. Tsarapkina Y.M., Yakubova E.Y. Web-Quest technology usage in professional self-determination // Rudn J. Informatiz. Educ. 2018. Vol. 15, No. 4. P. 373–381. DOI: 10.22363/2312-8631-2018-15-4-373-381 (In Russ.; abstract in Eng.)

18. Zakirova F.M., Qarshieva D.U. Quest for pedagogical technology and its use in education systems // Int. J. Integr. Educ. 2020. Vol. 3, No.V. P. 12–16. DOI: 10.31149/ijie.v3i5.369

19. Egorov S.Y., Sharonin K.A. Automated decision making in the problem solving of objects layout for chemical and refining industries using expert software systems // Chem. Pet. Eng. 2017. Vol. 53, No. 5–6. P. 396–401. DOI: 10.1007/s10556-017-0353-3

20. Nemtinov V., Bolshakov N., Nemtinova Yu. Automation of the early stages of plating lines design // MATEC Web Conf. / ed. Bratan S. et al. 2017. Vol. 129. P. 01012. DOI: 10.1051/matecconf/201712901012

21. Smorkalov A., Morozov M., Fominykh M. Collaborative work with large amount of graphical content in a 3D virtual world: Evaluation of learning tools in vAcademia // 2013 International Conference on Interactive Collaborative Learning (ICL). IEEE, 2013. P. 303–312. DOI: 10.1109/ICL.2013.6644587

22. Sheinbaum V.S. Interdisciplinary Activity training in virtual engineering environment: An actual state and prospects // Vyss. Obraz. v Ross. = High. Educ. Russ. 2017. Vol. 217, No. 11. P. 61–68. (In Russ.; abstract in Eng.)

23. Filatova M.N., Sheinbaum V.S., Shchedrovitsky P.G. Ontology of teamwork competency and approaches to its development at engineering university // Vyss. Obraz. v Ross. = High. Educ. Russ. 2018. Vol. 27, No. 6. P. 71–82. (In Russ.; abstract in Eng.)

24. Mikhaylyuk M.V., Torgashev M.A. Modeling and visualization of 3D virtual consoles in simulators // Sci. Vis. 2014. Vol. 6, No. 4. P. 50–60. (In Russ.; abstract in Eng.)

25. Tikhomirov G. et al. NRNU MEPhI Experience in development and application of visualization software in nuclear power plants education // Sci. Vis. 2012. Vol. 4, No. 2. P. 57–63. (In Russ.; abstract in Eng.)

26. Krol O., Sokolov V. 3D modelling of angular spindle’s head for machining centre // J. Phys. Conf. Ser. 2019. Vol. 1278, No. 1. P. 012002. DOI: 10.1088/1742-6596/1278/1/012002

27. Pozdneev B. et al. Digital transformation of learning processes and the development of competencies in the virtual machine-building enterprise environment // J. Phys. Conf. Ser. 2019. Vol. 1278, No. 1. P. 012008. DOI: 10.1088/1742-6596/1278/1/012008

28. Unal A., Karakuş M.A. Interacting science through Web Quests // Univers. J. Educ. Res. 2016. Vol. 4, No. 7. P. 1595–1600.

29. Nemtinov V., Zazulya A., Kapustin V., Nemtinova Yu. Analysis of decision-making options in complex technical system design // J. Phys. Conf. Ser. 2019. Vol. 1278, No. 1. P. 012018. DOI: 10.1088/1742-6596/1278/1/012018

30. Bolbakov R.G., Tsvetkov V.Y. Assessment of the quality of educational portals // Open Educ. 2017. No. 3. P. 22–28. DOI: 10.21686/1818-4243-2017-3-22-28 (In Russ.; abstract in Eng.)

31. Yarramreddy A., Gromkowski P., Baggili I. Forensic analysis of immersive virtual reality social applications: A primary account // 2018 IEEE Security and Privacy Workshops (SPW). IEEE, 2018. P. 186–196. DOI: 10.1109/SPW.2018.00034

32. Bigscreen. Your ultimate virtual reality hangout [Electronic resource]. 2022. URL: https://www.bigscreenvr.com/.

33. Hidayah N.A., Subiyakto A., Setyaningsih F. Combining webqual and importance performance analysis for assessing a government website // 2019 7th International Conference on Cyber and IT Service Management (CITSM). IEEE, 2019. P. 1–6. DOI: 10.1109/CITSM47753.2019.8965408

34. Google. Google forms: Create your own online surveys for free [Electronic resource]. 2021. URL: https://www.google.ru/intl/ru/forms/about/.

35. Djenno M., Insua G.M., Pho A. From paper to pixels: using google forms for collaboration and assessment // Libr. Hi Tech News. 2015. Vol. 32, ¹ 4. P. 9–13. DOI: 10.1108/LHTN-12-2014-0105

36. Assessment of the quality of an informational educational resource [Electronic resource]. 2022. URL: https://forms.gle/buDHWnZnG8WZMRKaA.