Towards Understanding of eSports Athletes' Potentialities: The Sensing System for Data Collection and Analysis

TL;DR

This paper introduces a sensing system for collecting and analyzing data from professional eSports athletes, demonstrated through a case study on gaze data from a CS:GO team, aiding in player assessment and scouting.

Contribution

It presents a novel sensing platform for real-time data collection from eSports athletes and provides a case study comparing amateur and professional gaze behavior.

Findings

Gaze data collected from professional and amateur players shows distinct patterns.

The sensing system enables detailed analysis of athlete performance.

Results support improved scouting and training methods in eSports.

Abstract

eSports is a developing multidisciplinary research area. At present, there is a lack of relevant data collected from real eSports athletes and lack of platforms which could be used for the data collection and further analysis. In this paper, we present a sensing system for enabling the data collection from professional athletes. Also, we report on the case study about collecting and analyzing the gaze data from Monolith professional eSports team specializing in Counter-Strike: Global Offensive (CS:GO) discipline. We perform a comparative study on assessing the gaze of amateur players and professional athletes. The results of our work are vital for ensuring eSports data collection and the following analysis in the scope of scouting or assessing the eSports players and athletes.

Equations

$$p_k^n=\frac{1}{{\tau }_n}\sum _{\tau =1}^{{\tau }_n}{p}_{\tau ,k}^n$$

$$[p_{\tau ,1}^n,\dots ,p_{\tau ,K}^n],\tau =1,\dots ,{\tau }_n.$$

Full text

I Introduction

Video gaming has steadily evolved from home entertainment towards the professional eSports industry within the past few years [1]. Indeed, eSports has been already recognized as a professional sport in many countries worldwide. Its global audience reached 380 million in 2018 and will reach more than 550 million in 2021 [25].

eSports is an organized gaming where the single players or teams compete against each other to achieve a specific goal by the end of the game. For the sake of understanding the player’s performance and for conducting the analysis of game statistics the professional players (or eSports athletes) often rely on online services, e.g. CS:GO Demos Manager (https://csgo-demos-manager.com) or a team coach. Online game services provide personal statistics while coaches perform the game analysis based on their experience rather than on science.

In terms of research, there are three relevant research areas in eSports (i) affective computing [10], (ii) prediction related research [3], and (iii) social structures in teams [30]. To date, the research in these areas is fragmented and lacks the experimental work with eSports athletes for data collection, modelling and interpretation tasks which could play a prominent role in the evolution of eSports research and make it practically feasible.

The emergence of Internet of Things (IoT) [24] and pervasive data science [13] paradigms enable us to instrument the physical environment with myriads of sensors and create an interconnected world generating huge volumes of heterogeneous data. IoT allows for connecting heterogeneous and autonomous sensors [5], systems and services in a global network enriched with intellectual capabilities [29]. It makes all ’things’ in the scope of IoT available any time from anywhere.

Data is getting a central tenant in computing applications where data collection, processing, and inference are of vital importance. Most of these ideas have been realized and deployed in the numerous smart-x applications including smart cities, smart transport, smart environment [32] and many others. These application are based on pervasive data science which focuses on data in pursuit of the concept of ubiquitous computing.

In this work we report on the sensing system for eSports athletes which enables the collection and processing of heterogeneous data. The proposed system relies on the IoT paradigm: we use heterogeneous sensors and data, as well as involve a game service for additional data collection. Also, we demonstrate a scenario how we collect the data from eSports athletes and perform their analysis for the assessment of professional quality.

This paper is organized as follows: in Section II we introduce the reader to the related works in the area, in Section III we present methodology used in this research. We describe experimental testbed and report on experimental results in Section IV. Finally, we provide concluding remarks and discuss our future work in Section V.

II Related Work

II-A Affective Computing

Since humans actively or subconsciously express affect in many different ways, there are also countless different approaches to detect signals of affect. They all begin with sensors that gather data about various aspects of the human subject. For example, a camera can be used to detect the facial expressions of a person or their posture, and a microphone can be used to record the person’s voice. The text they type could be another data source for affect recognition. There also exist many different sensors that record physiological data like the heart rate, the respiration rate, brain activity or sweating, to name a few. The choice of which of these sensor to use depends on factors such as the type of affect to detect, the desired availability of sensor data, and how much interference with the gameplay can be tolerated.

Affective Computing usually relies on sensors. When using Affective Computing in games, a first important task is to assess if a certain sensing technology can be used for a certain game and to select the most suitable sensors. Since there is no such thing as an optimal sensing technology, these decisions need to be made for each individual game [15].

Eye tracking is a process for detecting gaze position, pupil size, saccades and blinks [6]. Data analysis from eye-tracking studies has focused on synchronic indicators such as fixation (duration, number, etc.) or saccade (amplitude, velocity, etc.), rather than diachronic indicators (scanpaths or saliency maps) different metrics are available for comparing two scanpaths, using either distance-based methods (string edit technique or Mannan distance) or vector-based methods. Distance-based methods compare scanpaths only from their spatial characteristics, while vector-based approaches perform the comparison across different dimensions (frequency, time, etc.). These metrics are more or less complex and relevant depending on the situation to be analyzed [22]. According to [7] pupil’s response during affective picture viewing reflects emotional arousal associated with increased sympathetic activity. There are also methods suggested in [35] and [36] to measure cognitive workload and fatigue using eye tracking devices.

Emotional facial expressions represent the facial displays of emotions which determine different patterns of muscular correlates, cognitive responses, and brain activation. Autonomic and central nervous systems cooperate in order to provide a coherent pattern of mimic responses to specific contextual cues [4]. So there are computer vision methods to detect emotions. There are several emotion detection techniques discussed in, for instance, [26, 27].

Heart Rate Variability (HRV) tends to decrease with an increase in mental effort, stress, and frustration. Using machine learning, these emotional states and workload changes can be identified automatically [16]. The Blood Volume Pulse (BVP) is linked to the heart rate and can be used for emotion or mental stress recognition[15]. The electrodermal activity, according to [14, 33], could indicate awareness, the level of trust, stress or cognitive load.

II-B Metrics

In [14, 23] the authors described general terms and measures for metrics and analysis. The most important terms for the data are auditobility, entry event distribution, exit event distribution, virality, engagement and retention rate. There are several important game metrics. The first one are user metrics. These are metrics related to the people, or users, who play games, from the dual perspective of them: being either customers, i.e. sources of revenue, or players, who behave in a particular way when interacting with games. The second one is a performance metrics. These are metrics related to the performance of the technical and software-based infrastructure behind a game, notably relevant for online or persistent games. Process metrics are metrics related to the actual process of developing games.

II-C Gameplay input analysis

The gameplay input data that can be received from several sensors is extremely important for describing the gamer manner of playing and the level of his experience. Along with high-level game events such as deaths, kills, jumps, weapon fires the direct raw input data from keyboard, mouse and other sensors can provide more prolific insights. The analysis of mouse movements and buttons pressings as well as keyboard usage behaviour can be considered as individual characteristics of each player, particularly for predicting his skill level [8]. However, it requires proficient feature engineering with the raw data. The mouse input data can be described in terms of features such as the mean and the STD of the player mouse path, the mean and the STD of the velocity, the mean of click duration (the time that the button was pressed) and the mean number of clicks per some period of time. Also, the distribution of click intensity over screen zones can be considered [19]. Mouse movements of $x$ and $y$ positions are useful features as well. Regarding keys features the duration the special control keys were pressed, the most frequently pressed keys and the combinations of keys, pressed together, were used [8]. The gaze data from eye-tracker is also fruitful for analysis. It allows to classify the player skills and the patterns in player behavior [12, 11].

III Data Collection Platform

III-A Platform Design

In this section we describe the proposed IoT platform for the eSports data collection and analysis. The platform block diagram is shown in Figure 1. It includes three units: sensing, game PC and server, and allows the data collection from heterogeneous sensors and game service deployed on the CS:GO server.

In terms of sensing, the proposed platform contains the following sensors: EyeLink eye-tracker, Garmin Heart Rate Monitor (HRM) belt, key/mouse logger, in-game data logger (CSGO HLTV demo). These sensors are used as they inherently allow for unobtrusive sensing of players and do not posses a high number of artefacts like, e.g. EEG.

We used the EyeLink eye-tracking software for eye-position measurements and developed a custom script for capturing the data-stream from EyeLink library and recorded this data to the local csv text file. We measure the gaze position 30 times per second.

The wireless HRM sensor transmits the data using the ANT ultra low-power protocol 4 times per second. This frequency allows us to measure the time of each heart beat since the pulse of a person is obviously less than 240 beats per minute. Upon receiving the data from the HRM sensor, we calculate the average number of heartbeats per minute and transmit this value to the PC using UART protocol. USB-UART adapter and virtual COM port, as well as the developed program, was used to receive the HRM data, get the current time and write the received data to a file.

Also, we developed a custom script for capturing mouse position and mouse/keyboard pressed keys. The data are collected every 10 ms and stored locally on the PC.

PC unit is an advanced (storage, processing capability) gaming PC, where all external physiological sensors, i.e. eye tracker and HRM, are connected. Apart from the physiological data we perform logging of mouse and keyboard on the PC, as well as collection of game statistics. All required interfaces, libraries and loggers are installed on the game PC which acts as an experimental testbed (see Figure 2).

The data from all the sources are stored on the PC except for the ’Demo’ files which are recorded on a game server and then are copied to the PC right after each game session. We record the in-game data during the game using the HLTV bot.

IV Methodology

IV-A Objectives

It is essential for a coach and for every player interested in improving his skills to perform the analysis of the performance during both competition and training games. In most cases currently available tools provide analytics for the in-game data, e.g. a player accuracy based on the total number of weapon shots and the area of successful hits. Using wearable sensors we can assess a number of other metrics from game and evaluate the overall athlete ’quality’. The combination of those metrics could reveal weak and strong points of each athlete. The data has to be collected and preprocessed before the evaluation.

IV-B Game Scenario

There are two roles for the Retake modification of CS:GO discipline: a terrorist team and counter-terrorist team. The terrorist team (2 players) is characterized by the defensive role. They have a bomb planted on the territory and have to defend it. The counter-terrorist team (3 players) has to deactivate the bomb or to kill the enemy team. The system shows the bomb location on the map at the beginning of each round. Players are also asked to buy exactly the same set of weapons for each round. Each round lasts for 40 seconds while there are 12 rounds in total. This scenario must be played without any breaks between the rounds.

IV-C Participants

In this experiment players with different CS:GO experience were invited. Some players have not had CS:GO experience at all, while other players have spent impressive amount of time in this discipline ($>$10,000 hours), which means they play more than 4 hours per day. In total 15 players (5 professional and 10 amateur) participated in our experiment.

Prior to participate in the experiment, all participants were informed about the project, its goals, and the experiment. We obtained a written consent from each participant in the experiment. Afterwards, we ensured by a questionnaire that all the participants are in a good form and do not take any drugs in order to avoid the interference with the experimental results.

V Evaluation

The platform we propose in this work allows for monitoring of the crucial parameters of an eSports athlete during the game. In this section we discuss our analysis in details.

V-A Gaze Analysis

In this section, we provide the analysis of the gaze of professional athletes and amateur players. We state and discuss several important differences between the athletes and players.

During the Counter Strike game the cyber atheltes have to visually control several in-game elements. First, they have to track the dynamic 3D game environment (enemies, allies, shots, positions, etc.). Second, the game player has to regularly check the game User Interface (UI) that overlay the game 3D environment. We show these UI elements in Figure 3.

The UI elements include the health & armor panel, weapon & ammo panel, in-game radar, kill & death log, game timer & player list. Controlling them is crucial indeed: the radar provides the player with the information about the teammates and enemy position, weapon & ammo panel allows for efficient management of ammo usage, etc.

The gaze data set consists of 2-dimensional time series of moving screen coordinates $(x,y)$ for each of 15 players (5 professional and 10 amateur). Each time series is recorded at the frequency of 60 Hz and contains 10 repeated rounds lasting approximately 1 minute each. The gaze positions are within 1920x1080 screen size.

To pre-process the data we first extract the periods when the player was alive. To do this we used the parsed information from the game demo log file. Next, we performed the linear interpolation of missing values of players’ gaze. It was performed only for the gaps with missing data, that are less than $0.1$ seconds (6 missing consequent observations maximum). The total amount of missing data is about $4\%$ (situations when the eye-tracker looses player’s gaze or player moves the gaze away from the monitor).

We denote the resulting prepossessed time series for the player $n=1,\dots,15$ by $(x_{t}^{n},y_{t}^{n})$, where $t=1,\dots,t_{n}$ is the integer index of time. We assume that the gaze positions $(x,y)$ are mainly caused by the specific zone of interest shown on the screen during the game (see Figure 3). By confirming this assumption one can then match the gaze position with a single categorical value representing such an area of interest, e.g. radar, weapon panel or aiming cross-hair.

We apply $k$-center clustering to the set of all observed gaze positions of all players $(x_{t}^{n},y_{t}^{n})$ (for all $n,t$). We manually set $k=9$ cluster centers and use Euclidean metric as a distance. The cluster centers are given in Table 4 below.

The obtained clustering in shown in Figure 5 below. Six of the clusters centers represent different UI-related zones of player’s interest shown in Figure 3. Zone $1$ is the aiming cross hair, Zone $2$ is the radar area, Zone $3$ is the health & armor panel, Zone 5 is the weapon & ammo panel, Zone 7 is the kill & death log, Zone 9 is the game timer & player panel. We also add three zones around the aiming cross-hair that represent players gaze concentrated near the aiming cross-hair (right, left and below the cross-hair for Zones 4, 6 and 8 respectively).

By considering 15s rolling window (indexed by ${\tau=1,\ldots,\tau_{n}}$) we calculate the probabilities over all clusters $p_{\tau,k}^{n}$, $\sum_{k=1}^{K}p_{\tau,k}^{n}=1$, as the frequency when the gaze of $n$-th player was focused on the $k$-th zone of interest. Also, we consider similar distribution averaged over all rolling windows

[TABLE]

as a measure of comparative usage of the gaze zones for each player.

We then raise the question whether there is a difference in the gaze of the professional athletes and amateur players and whether they can be clearly explained by simple rules. To answer this question we apply the Principal Component Analysis (PCA) [34, 18] to the set of all $K$-dimensional vectors of distributions

[TABLE]

The projection of such data onto the linear subspace of the first two principle components (sorted by their variance) is shown in Figure 6. Green points correspond to the gazes of amateur players and the red points correspond to the professional ones. Small points represent the projections of points (2) for some randomly chosen rolling windows $\tau$. Large points are the projections of averaged distributions (1).

The first main component explains $57\%$ of the variance in the data, while the second component explains only $13\%$. Nevertheless we can clearly see that professional athletes can be easily separated from amateur players by these two components. To answer why this happens we provide a more detailed analysis.

Surprisingly, it turns out that both main components are almost sparse with only one dominant coordinate. The dominant coordinate in the first main component corresponds to the Zone $1$ (aiming cross-hair zone) while the dominant coordinate in the second main component corresponds to the Zone $2$ (radar zone) (see Figure 5).

Based on this analysis, we conclude that the professional athletes spend more time looking at the screen center than the amateur players. At the same time athletes do not look at the radar too often as players do. We show the gaze heat map of an average professional athlete in Figure 7a and average amateur player in Figure 7b. The gaze of the professional athlete is most of the time concentrated at the screen center while the amateur players spent more time looking at the UI elements, especially the game radar.

Since Counter Strike discipline is the first-person shooter game, it was reasonable to expect that players’ gaze most of the time is concentrated around the screen center where the aiming cross-hair is located. The fact that the professional athletes spent more time looking at the aiming cross-hair is logical enough: they perfectly know the map and perfectly position themselves in the game — that is why there is no need to look around. This also explains the point why they do not need to use radar frequently.

V-B Mouse & Keyboard Usage Analysis

In this section, we report on the analysis of the mouse and keyboard usage by professional athletes and amateur players.

In the CS:GO discipline main control keyboard keys are common for most players:

W, S — forward and backward movement keys; 2. 2.

A, D — left and right movement keys; 3. 3.

Mouse 1 — weapon shooting key (left mouse button).

The way how the player uses the control keys has an impact on a player skill. For investigating this point we extract a number of features related to the usage of specific keyboard keys. In this section, we discuss the features that highly correlate with the specific skill of players and athletes.

First, we consider the period of time when the player used the A or D keys, i.e. movement to the right and to the left. Second, we examine the period of the time when the player pressed the shooting button MOUSE1 while moving forward W. We plot the distribution of these features on Figure 8 and Figure 9, respectively. We note here that we smooth the plots by using Gaussian Kernel Density Estimation.

On the one hand, we can clearly see that the professional athletes often use the left and right movement keys, i.e. A and D. At the same time, amateur players often combine the moving forward (running) key with shooting. According to the game engine this action significantly reduces the shooting accuracy. This analysis confirms that professional athletes better control their movements in game scenario.

VI Conclusion

In this work, we have demonstrated a platform for collecting the data from eSports athletes. The platform includes three types of sensors: physiological, environmental and game telemetry. Using this platform we carried out the experiment on the data collection from professional athletes and amateur players. The special emphasis was made on the gaze analysis.

Our future research aims at the data collection from the heterogeneous sensors and finding correlations between the physiological and psycho-emotional conditions of eSports athletes.

Analysis of eSports data requires the development of specialized machine learning methods: there are outliers due to faulty sensors, part of the data is represented in the form of multidimensional time series, and another part is provided as heterogeneous events streams, etc. Thus we should develop a unified methodology for eSports data processing, including pre-processing data to remove anomalies [2, 28, 17], methods of imbalanced classification [31, 9] and dimensionality reduction [21, 20] to extract specific patterns and predict rare events.

Acknowledgment

The reported study was funded by RFBR according to the research project No. 18-29-22077$\backslash$18.

Authors would like to thank Skoltech Cyberacademy, CS:GO Monolith team and their coach Rustam “TsaGa” Tsagolov for fruitful discussions while preparing this article. Also, the authors thank Alexey ”ub1que” Polivanov for supporting the experiments by providing a slot at the CS:GO Online Retake server 111http://ub1que.ru.