1. Introduction

Many tasks in Digital Humanities (DH) research involve the processing of the linked data, wherein the graph theory appears to be a powerful methodological and technological base for solving associated problems. Taking into account the specifics of DH, the considered data are normally quite big, but their handling requires human attendance and cannot be fully automated. One of the key means to help DH specialists to fulfill their everyday work is scientific visualization and visual analytics (VA) that allows to present related data in an observable interpretation-ready form. Our goal is to develop an ergonomic and flexible tool for graph-based visualization of interconnected data that allows comprehensive VA in DH research. The new high-level component for circular graphs’ visualization is presented to tackle data filtering problems and improve the cognitive power of visual analytics. High-level ontology-driven data extraction, transformation and loading (ETL) mechanism is proposed to enable obtaining the data from different sources and process them in a flexible way. The software developed is demonstrated by solving the problems from the applied linguistics domain.

2. Key Contributions

Along with the traditional tools for graph VA (like node search based on regular expressions, highlighting of incident edges and connected nodes, depicting clusters by colors, etc.) we propose the following key features:

1. Ontology-driven data ETL mechanism that allows obtaining the input data from different sources and preprocessing them by custom algorithms defined by means of high-level visual programming language.

2. Multilevel ring scale placed around the circular graph that allows grouping the graph nodes according to the given classifier and automatically reordering them at runtime.

3. Equalizing filter that allows applying different filtering thresholds to different groups of graph nodes/edges to cut off the noisy data, which noise has a non-uniform strength distribution across the graph.

4. Graph state calculator that allows data comparison by performing different operations (union, intersection, etc.) on the data slices shown within the graph.

5. Synchronization of the data slice currently visualized by the graph with the corresponding localized area on the geographical map (selecting a specific area on the map opens up the data slice on the graph that is linked to this area, and vice versa, choosing specific data slice on the graph highlights the corresponding area on the map).

3. Related Work

VA is no doubt a powerful methodology to conduct research in a field of DH, but, as indicated in [1], nowadays there is a noticeable talent gap between the VA scholars and digital humanists. While DH and VA have a huge potential of coevolution, the research results presented in the literature are typically valuable either only for DH, or only for VA, and rarely for both simultaneously [1]. This is because DH projects often lack researchers with deep computer science skills, and consequently have to rely on the existing general-purpose visualization tools, instead of driving the actual software development. But in that case, some tasks remain unsolved because of traditional software limitations [1, 2]. W. Huang et al. tackle this problem by proposing a so-called user-centered approach to the process of visualization making (graph-based visualization in particular). This approach ensures the creation of cognitive graphics tools, which development comprises design and evaluation stages [2]. On the design stage, “the designer applies design principles and chooses the visualization best supporting perception and cognition”, and on the evaluation stage “visualization is evaluated to understand how cognitive processes are affected” [2].

Similar, but slightly less formal approach is proposed by S. Jänicke, who describes an “ideal” VA+DH project as a close collaboration between the computer scientists and digital humanists, where each visualization feature proposed is immediately tested and validated in terms of its viability for DH research and then either approved for further development or rejected [1]. Working on our VA tools, we have chosen this exact strategy.

For graph visualization, the Gephi system is traditionally used [3]. Being feature-rich, this system, however, provides instruments for layout the graphs of free structure, while we found out that sometimes the circular graphs [4] are more comprehensive by depicting data sets. Moreover, as stated in [5], it is often desirable to have the graph visualization tools in a Web application, without installing additional software.

An important point of graph visualization is the data preparation stage. To ensure the intuitive and flexible data preparation process we suggest to declare its steps by data flow diagrams (DFDs) [6]. A lot of popular visualization software use such an approach, for example, Blender, Maya, Substance Designer, etc., so it proved its efficiency in terms of data processing and rendering pipeline declaration.

We use model-driven architecture based on the ontology engineering methods [7] to achieve the configuration flexibility and adaptation of the software to the specifics of the application domain without source code modification. We construct the ontologies within visual editor ONTOLIS [7].

4. Background

Our previous research work was dedicated to the development of ontology-driven scientific visualization and VA system called SciVi (https://scivi.tools) [8]. This system is portable across all the popular platforms (Windows, GNU/Linux, macOS, iOS, Android). It is organized as a client-server application, having both thick (native, written in C++ using Qt 5 framework) and thin (browser-based, written in TypeScript and JavaScript, utilizing HTML5 and CSS3) clients. The behavior of this system is fully controlled by underlying ontologies, which allow deep reconfigurations of SciVi, extension of its ETL and data rendering capabilities, whereby leveraging adaptation to the completely new visualization and analytics tasks without changing the source code of its core. Faced the problems in a field of DH during the case study of verbal and nonverbal behavior of social network users, we built the graph VA toolset upon the SciVi [9]. Tried out different graph layouts, we focused on the circular one because of its good readability [4].

We implemented a graph visualization SciVi component (called SciVi::CGraph) as a Web application in TypeScript utilizing PixiJS (https://www.pixijs.com) rendering engine. The graph nodes are uniformly distributed on a circle and the edges are drawn as quadratic Bézier curves with the control point in the circle’s center. Different slices of data can be displayed on the same graph using a scale of states that allows fast switching between them. Data slices can be organized in a hierarchy, therefore this scale supports multiple levels. Examples of different graphs can be found online: https://graph.semograph.org/cgraph/.

SciVi tools have been integrated into the Semograph information system [10]. Semograph is aimed to solve different DH tasks involving methods of computational linguistics by supporting a wide range of operations on the textual content, including tagging, classification of terms, building semantic relations, etc. The integration with SciVi allowed us to utilize advanced visualization features including the rendering of graphs.

5. SciVi ETL Mechanism

The conceptual scheme of the data processing within the SciVi system is shown in Fig. 1.

 

Fig. 1. Data processing pipeline within SciVi

 

Currently, CSV format is used to transfer data from Semograph into SciVi, since export to this format is natively supported by Semograph. However, it is easy to switch to any other data representation since the ETL mechanism of SciVi is very flexible. This mechanism is implemented within the SciVi Data Processing Module and driven by the ontological knowledge base. Underlying ontologies describe different data formats and data interpretation rules, as well as available data preprocessing filters and data visualization techniques. Thanks to this, changing or extending these ontologies is enough to alter SciVi behavior adapting it to the new VA tasks. But the changing of ontologies requires knowledge engineering skills, thereby is unwanted for the end-users and is dedicated to the system administrator.

The end-users are provided with a more high-level steering instrument: Data Flow Editor. This SciVi module is based on the Rete (https://rete.js.org/) JavaScript framework and implements a graphical user interface (GUI) to compose a data processing algorithm from the high-level building blocks utilizing DFDs. The example of DFD describing the extraction of data from an arbitrary CSV file is shown in Fig. 2.

 

Fig. 2. DFD defining ETL and data visualization in SciVi

 

Each node in the DFD represents a particular step in data obtaining, processing or visualization. For example, “CSV Table” defines file reading; “Get Range” allows to specify the subset of values within the CSV table; “Make Nodes” sets up the composition of the graph nodes internal representation; “Make Graph Data” corresponds to the stage of interconnecting the graph nodes with weighted edges; “Circular Graph” defines the data rendering using SciVi::CGraph VA component. Links between DFD nodes depict the data flow and their color is bound to the type of transmitting data.

The set of available DFD nodes’ types correspond to the set of operations on the data available in SciVi. It is constructed automatically according to the underlying ontology and presented to the user as a toolbar palette. Each data processing operation has its own description that may be altered or extended to change the actual behavior of the entire system. For example, the ontology fragment describing “CSV Table” is shown in Fig. 3.

 

Fig. 3. Fragment of SciVi ontology describing CSV table reader

 

It can be seen that the “CSV Table” node is treated as a data source, has CSV file as a setting parameter and table of values as an output. The inputs and outputs are drawn as round sockets in Fig. 2. Settings are not presented in the Fig. 2, since they appear in a special context menu within the SciVi DFD editor, when the users clicks on the corresponding DFD node.

The actual implementation of this data reader is described by the “CSV Table Worker” concept in the ontology. This concept has an internal attribute (not drawn in the figure, since the figure shows concepts and relations only) with a link to the JavaScript code fragment that implements CSV reading with help of PapaParse (https://www.papaparse.com) library. An important part of the “CSV Table” description is the “ClientSideWorker” concept. It identifies that the reading and parsing takes place within the browser (on the client side), without sending the data to the SciVi server. Although the SciVi architecture allows server-side processing, currently the amounts of data we faced in our tasks were small enough to be handled locally.

The ontology fragment describing “Circular Graph” is presented in Fig. 4.

 

Fig. 4. Fragment of SciVi ontology describing circular graph VA component

 

“Circular Graph” is a visual object that has two inputs: “Data” (“GraphData” type) and “Classifier” (“Grid” type), as well as two settings: “Title” (“String” type) and “SortNodes” (“Bool” type). “Data” represents the actual data set that is to be visualized as a graph. “Classifier” represents the dictionary that defines groups for the graph nodes. Then the groups are represented by a special ring scale (see Section 6.1). “Title” is a user-defined string that is placed on top of the graph during the visualization. “SortNodes” is a user-defined flag that switches whether graph nodes should be ordered lexicographically, or their order should be retained from the input data.

In the ontology, the client-side implementation of “Circular Graph” is presented by the node named “CGraph”. This implementation uses the SciVi::CGraph library written in JavaScript. It is worth noting that this library is also a part of the OpenSource SciVi project: https://github.com/scivi-tools/scivi.graph. The distinctive features of SciVi::CGraph are described in details in the next section.

6. Visual Analytics SciVi Components

While the SciVi platform contains different VA components, in this paper we focus on the SciVi::CGraph. SciVi::CGraph VA component accepts the data in JSON representation. Once the user has created the DFD for the particular task and started the visualization, this component renders the graph and provides its own GUI allowing different interactions with that graph, including zooming, panning, nodes and edges selection, data filtering, etc. The most important distinctive features of SciVi::CGraph are described below.

6.1. Multilevel Ring Scale

In case, when a classification of graph nodes is defined, SciVi::CGraph draws a special ring scale around the graph to visually highlight the given nodes’ classes. The number of rings in this scale is potentially unlimited, so the nodes’ classifier can have multiple levels. A special tree view in a sidebar of the graph allows to explore the classifier and switch the visibility of nodes belonging to individual classes. Colors of the ring sectors, which depict the classes, can be assigned manually, but also set automatically based on the special heuristic algorithm that maps the classifier’s hierarchy to the HSV color model in a way the neighbor ring sectors have distant colors to be visually distinguishable.

To evaluate different hypotheses, the user can change the order of scale rings by drag and drop, command the graph to sort the nodes accordingly and set the color of nodes to the color of any ring sector they belong to. These interactions help to find out, which order of hierarchy levels is the most meaningful one in terms of structuring the interconnected data.

Fig. 5 shows the results of the correlation analysis of 38 topics extracted from 48 stories told by informants as self-presentation [11]. The interactive graph is available online: https://graph.semograph.org/cgraph/aboutmyself/index.html. The sample of informants is balanced by sex, age, and education level. Graph nodes depict self-presentation texts, edge thickness represent correlations coefficients (all correlations are positive; all coefficients below 0.8 are filtered out). Social (education level: secondary, higher) and demographic (sex, age group) parameters are shown on the ring scale groping the nodes accordingly. The groups are nested according to the order of the rings.

 

Fig. 5. Correlation of topics in self-presentations, grouped by age (a), education (b), and sex (c)

 

The aim is to find out, which parameter dominates by grouping the informants together. Related to DH it means to find, which social/demographic informant groups consolidate more by talking about themselves. Related to the graph theory it means to find, which layout of nodes provides their better clustering. The proposed mechanism of the ring scale reordering allows quick checking of different variants and inspecting them visually. While Fig. 5a (topmost grouping by age) and 5b (topmost grouping by education level) look messy, Fig. 5c reveals significant dense “community”, corresponding to the stories told by females (at the same time, there is almost no correlation between males’ stories). Further interpretation of this material is outside of this paper’s scope, but the corresponding milestone of related DH research is considered to be reached. It is worth noting, that it took less than a minute to find this solution using SciVi::CGraph.

6.2. Equalizing Filter

Sometimes the noisy data on a single graph may have a non-uniform distribution of the noise strength. In this case, filtering the entire data set with the single threshold appears to be meaningless and threshold adaptivity is required. We often face this problem in multipartite graphs comprising interconnected data of different nature, or data, which parts were differently preprocessed. To tackle this problem, we propose a so-called equalizing filter that can have individual parameters for selected groups of nodes and edges (resembling the sound equalizer that can differently affect selected parts of the spectrum).

Currently, the equalizing filter within SciVi::CGraph operates as a set of range-based cutoff functions tied to the ring scale. By default, there is one global cutoff function (affecting the entire graph), but, if needed, the user can add auxiliary local ones for any sector of the ring scale. If a node or an edge is affected by multiple cutoff functions (global one and multiple local ones according to the hierarchy of the ring scale), their ranges are intersected to build the resulting filter. Node or edge is filtered out if its weight lays outside the functions’ range intersection.

The practical use case of the equalizing filter is demonstrated in Fig. 6.

 

Fig. 6. Relationships between the verbal behavior of social network users and their psychological characteristics

 

This figure represents the filtered data of the relationships between the verbal behavior of social network users (SNUs) and their psychological characteristics. The interactive graph is available online: https://graph.semograph.org/cgraph/psycho_reduced/index.html. The psychological parameters are obtained by two questionnaires (personality features and self-esteem) [12] fulfilled by the sample of SNUs. The verbal behavior is revealed with the help of the linguistic analysis from the comments written by these users in social networks. The filtering is individual for each psychological parameter because each of them has its own statistical features (minimum, maximum, average, standard deviation). This approach allows leaving only the dominant indicators for each psychological parameter. Fig. 6 demonstrates, that after equalizing the indicators, it can be revealed that the SNUs of the female gender, who use obscene words in the public social network space, are characterized by low self-esteem, low conscientiousness, low agreeableness, high neuroticism, middle openness, and low extraversion.

6.3. Graph State Calculator

To visually compare the structure of data slices displayed in the graph, we implemented a special graph state calculator. It allows to perform a sequence of basic set operations on the graph states: union, intersection, difference, and symmetric difference.

Fig. 7 demonstrates the states of “Moscow” geoconcept. The interactive graph is available online: https://graph.semograph.org/cgraph/geoconcepts_reduced/index.html. In this research, under the term “geoconcept” we understand a set of collective opinions about a geographical object. These opinions can be revealed from the associations people come up with [13]. Graph nodes represent the semantic categories of associations (extracted according to the special classifier within Semograph system), edges identify the co-presence of linked categories in the analyzed associates (derived from a group of informants). The actual structure of geoconcept presented as a set of association categories depends on the region. In this experiment we collected 3 datasets: in Perm (Fig. 7a), Biysk (Fig. 7b) and Orenburg. The state scale (drawn below the graph) provides quick navigation between these data sets and makes it possible to visually compare them. However, to make this comparison more elaborated and meaningful, set operations can help. As an example, Fig. 7c shows the intersection of Perm and Biysk data sets, allowing to view their common parts.

 

Fig. 7. States of “Moscow” geoconcept as viewed in Perm (a) and Biysk (b) along with their intersection (c)

 

Thus, the graph state calculator provides a good basis for conducting comparative DH studies and facilitates the process of interpreting research results.

6.4. Synchronization with the Geographical Map

The data obtained during the DH research are often linked to the particular geographical regions, for example, countries, cities, or even to the specific addresses. Cognitive representation of these data for the sake of visual analytics involves geographical maps. However, according to Schneiderman’s Mantra “Overview First, Zoom and Filter, Details on Demand” [14], the simple placing of markers on the map is not enough for the full-fledged comprehensive visual analytics. While the “Overview”, “Zoom”, and “Filtering” are normally achievable on the map itself, the last but not least part of getting “Details on Demand” often requires additional visualization and analytical means.

In our practice we had to work with the slices of interconnected data, where each slice is linked to the specific geographical area (represented as a set of points with geographical coordinates). Each slice can be represented by the SciVi::CGraph, but the distribution of corresponding points should be visualized on the map to allow adequate overview and zooming. To follow the workflow defined by Schneiderman’s Mantra, we came up with synchronizing SciVi::CGraph and map in a way, that selecting the specific area on the map opens up the corresponding graph and vice versa, choosing specific data slice on the graph highlights the corresponding area on the map. This synchronization provides the user with the ability to get the detailed information about the interest region on demand.

To enable this kind of synchronization, we slightly changed the ontology-based description of “Circular Graph” VA component, appending a new input and a new output responsible for receiving and transmitting the data slice selection. The updated ontology fragment describing the “Circular Graph” is shown in Fig. 8. Nodes retained from the original ontology (as shown in Fig. 4) are filled white, relations retained from the original ontology are drawn dotted. Newly introduced nodes are highlighted yellow, newly added relations are drawn solid.

 

Fig. 8. Extended ontology describing circular graph VA component

 

As seen in the figure, it is just a matter to enrich the ontology with 3 new nodes and 6 new relations to add the new functionality to SciVi without any source code modification of the SciVi core. “Select State” and “State Selected” describe string messages that are received and transmitted respectively by the graph VA component.

This ontology change has two major effects within SciVi. The first one is a change of “Circular Graph” DFD node interface. Fig. 9a depicts the “Circular Graph” DFD node as it appeared before ontology was being modified, and Fig. 9b reflects the update. As it can be seen, “Circular Graph” got two additional sockets, the input one called “Select State”, and the output one called “State Selected”.

 

Fig. 9. Comparison of circular graph DFD node interface before (a) and after (b) ontology modification

 

The second effect of ontology modification relates to the “Circular Graph” VA component internals. Two variables are automatically generated and are accessible via special internal dictionaries. First one is read-only and accessible via “Selected State” tag; second one is write-only and accessible via “State Selected”. These variables serve to pass the messages about the state changes. The related message passing source code is automatically generated. The only thing we had to append manually are four lines of code within “Circular Graph” VA component JavaScript implementation (the SciVi core remains unchanged):

if (HAS_INPUT["Select State"] && INPUT["Select State"])

    graph.selectGraphState(INPUT["Select State"]);

else if (graph.stateChanged())

    OUTPUT["State Selected"] = graph.currentState();

As a result, graph VA component became capable of synchronizing its selected state (shown slice of data) with other VA components, for example, the ones implementing the geographical maps. Further, if the corresponding nodes or relations are removed from the ontology, whereby the “Selected State” and “State Selected” variables will not exist anymore, there is no need to revert the changes in the code of VA plugin: the internal SciVi mechanisms take care of eliminating the “dead code” automatically at runtime. So, enabling and disabling the state selection functionality can be now performed via ontology modification only.

To visualize the interactive map, we integrated SciVi with the Leaflet (https://leafletjs.com) library. This library is easy to use and suits well for lightweight 2D scientific visualization of geo-tagged data, despite it supports no hardware acceleration of rendering [15]. We described two new visual components in the SciVi ontology: “Map” (that is responsible to visualize the map using Leaflet) and “Map Markers” (that accepts a table of objects with their properties, including geographical coordinates, turns them into a set of Leaflet map markers, and manages the map selection).

One of DH use cases in the area of cultural geography that utilizes the means described above is devoted to the study of the meaning of slogans transmitted between the people in the informal urban space (taking into account different districts, such as central districts, sleeping quarters and suburbs). The social slogans (521 inscriptions on walls, fences and other street objects) in the urban space of Perm were collected from different districts of the city and then categorized with help of Semograph system by genre, topic, lexic, writing method and other characteristics. The aim of VA tools is to depict the slogans on the city map and the results of their categorization into the groups with the same semantics. The DFD composed of the updated “Circular Graph” node and corresponding map-related nodes is shown in Fig. 10.

 

Fig. 10. DFD describing the synchronized graph and map VA components

 

The visualization result that corresponds to the DFD from Fig. 10 is shown in Fig. 11. As it can be seen, when multiple visual objects appear in the DFD, SciVi automatically applies tile-based layout to place the corresponding views on the screen all at once. This feature enables simultaneous analysis of different visual models.

 

Fig. 11. Distribution of slogans in the Perm city synchronized with the graph depicting slogan characteristics grouped by districts

7. Conclusion

Thanks to the features discussed, SciVi::CGraph allows advanced interactive VA of interconnected data in DH, harnessing the power of visual analytics and Data Science. According to the feedback from the DH researchers of Perm State University, these tools outperforms traditional graph analysis software like Gephi in the tasks, which require special analytics features. Like SciVi VA system, SciVi::CGraph is an open-source project licensed under the terms of GPLv3: https://github.com/scivi-tools/scivi.graph.

SciVi::CGraph is being iteratively developed in close collaboration with DH specialists and each new feature is immediately evaluated in real-world research projects (in exact accordance with the cooperation model described in [1]). For example, SciVi::CGraph was used by exploring the egocentric field of speaker in the macedonian language [16], in the study of social network users’ speech in the research project of Perm State University [17], and by the semiotic analysis of geomental maps [13]. Also, SciVi::CGraph was utilized in the Sirius education center in the project “Images of Large Russian Cities in the Linguistic Consciousness of Senior Schoolchildren” [18].

Taking into account the needs of conducted DH research, we plan to extend our scientific visualization system SciVi with new feature-rich visualization components for free structure graphs and graphs with volumetric 3D layout. Next, we plan to test the scalability of our tools handling different massive data collections.

8. Acknowledgements

The reported study is supported by Russian Science Foundation (grant number 20-18-00336).

We would like to thank Elena Erofeeva, Ekaterina Khudyakova, Natalia Zelianskaia and Tatyana Surnina for help in data assembling and processing, as well as for the valuable feedback about the SciVi::CGraph usability.

References

1. Jänicke, S. Valuable Research for Visualization and Digital Humanities: A Balancing Act // Workshop on Visualization for the Digital Humanities, IEEE VIS 2016, Baltimore, Maryland, USA. – 2016.

2. Huang, W., Luo, J., Bednarz, T., Duh, H. Making Graph Visualization a User-Centered Process // Journal of Visual Languages and Computing. – 2018. – Vol. 48. – PP. 1–8. DOI: 10.1016/j.jvlc.2018.07.001.

3. Grandjean, M. Introduction to Network Visualization with GEPHI [Electronic Resource]. – 2013. URL: http://www.martingrandjean.ch/introduction-to-network-visualization-gephi/ (last accessed: 17.10.2020).

4. Ageev, A. A Triangle-Free Circle Graph with Chromatic Number 5 // Discrete Mathematics. – 1996. – Vol. 152. – PP. 295–298. DOI: 10.1016/0012-365X(95)00349-2.

5. Tominski, C., Abello, J., Schumann, H. CGV – An Interactive Graph Visualization System // Computers & Graphics. – 2009. – Vol. 33, I. 6. – PP. 660–678. DOI: 10.1016/j.cag.2009.06.002.

6. Lee, B., Hurson, A.R. Issues in Dataflow Computing // Advances in Computers. – 1993. – Vol. 37. – PP. 285–333. DOI: 10.1016/S0065-2458(08)60407-6.

7. Chuprina, S., Nasraoui, O. Using Ontology-based Adaptable Scientific Visualization and Cognitive Graphics Tools to Transform Traditional Information Systems into Intelligent Systems // Scientific Visualization. – 2016. – Vol. 8, No. 3. – PP. 23–44.

8. Ryabinin, K., Chuprina, S. High-Level Toolset for Comprehensive Visual Data Analysis and Model Validation // Procedia Computer Science. – 2017. – Vol. 108. – PP. 2090–2099. DOI: 10.1016/j.procs.2017.05.050.

9. Ryabinin, K., Belousov, K., Chuprina, S., Shchebetenko, S., Permyakov, S. Visual Analytics Tools for Systematic Exploration of Multi-Parameter Data of Social Web-Based Service Users // Scientific Visualization. – 2018. – Vol. 10, No. 4. – PP. 82–99. DOI: 10.26583/sv.10.4.07.

10. Belousov, K., Erofeeva, E., Leshchenko, Y., Baranov, D. “Semograph” Information System as a Framework for Network-Based Science and Education // Smart Innovation, Systems and Technologies. – 2017. – Vol. 75. – PP. 263–272. DOI: 10.1007/978-3-319-59451-4_26.

11. Pavlova, D., Garanovich, M. Variability of Semantic Structure of Oral Spontaneous Monologues “About Myself” depending on “Gender” Factor // Nauchnyi dialog. – 2019. – I. 5. – PP. 107–122. DOI: 10.24224/2227-1295-2019-5-107-122.

12. Shchebetenko, S. Reflexive Characteristic Adaptations Explain Sex Differences in the Big Five: But not in Neuroticism // Personality and Individual Differences. – 2017. – Vol. 111. – PP. 153–156. DOI: 10.1016/j.paid.2017.02.013.

13. Zelyanskaya, N., Belousov, K., Ichkineeva, D. Naive Geography and Geopolitical Semiotics: the Semiotic Analysis of Geomental Maps of Russians // Semiotica. – 2017. – Vol. 2017, I. 215. – PP. 235–253. DOI: 10.1515/sem-2016-0231.

14. Shneiderman, B. The Eyes Have It: a Task by Data Type Taxonomy for Information Visualizations // Proceedings 1996 IEEE Symposium on Visual Languages. – 1996. DOI: 10.1109/VL.1996.545307.

15. Vorobev, A.V., Vorobeva, G.R. Web-Oriented 2D/3D-Visualization of Geomagnetic Field and Its Variations Parameters // Scientific Visualization. – 2017. – Vol. 9, No. 2. PP. 94–101.

16. Boronnikova, N., Taleski, A., Belousov, K., Ryabinin, K. Visual Representation of the Egocentric Field of the Speaker in the Macedonian Language (Experimental Study) // Perm University Herald. Russian and Foreign Philology. – 2018. – Vol. 10, I. 3. – PP. 13–27. DOI: 10.17072/2037-6681-2018-3-13-27.

17. Belousov, K., Erofeeva, E., Baranov, D., Zelyanskaya, N., Shchebetenko, S. The Multi-Parameter Analysis of Linguistic Data in the Information System Semograf (On the Example of the Study of Social Network Users’ Speech // Tomsk State University Journal of Philology. – 2020. – I. 64. – PP. 6–29. DOI: 10.17223/19986645/64/1.

18. Linguists of Perm University Became Curators of Scientific Research at the Educational Center “Sirius” [Electronic Resource] // PSU. – 2019. URL: http://www.psu.ru/news/lingvisty-permskogo-universiteta-stali-kuratorami-nauchnogo-issledovaniya-v-obrazovatelnom-tsentre-sirius (last accessed 17.10.2020).