What Makes a Good Team? A Large-scale Study on the Effect of Team Composition in Honor of Kings

TL;DR

This large-scale study analyzes how team composition affects effectiveness in Honor of Kings, revealing that diversity improves performance and tenacity but can increase abuse, with implications for team dynamics and prediction models.

Contribution

The paper provides the first large-scale empirical analysis of team composition effects in a popular MOBA game, highlighting diverse teams' strengths and weaknesses.

Findings

Diverse teams perform well and show tenacity.

Diverse teams are more prone to abuse when losing.

Features based on team composition can predict team outcomes.

Abstract

Team composition is a central factor in determining the effectiveness of a team. In this paper, we present a large-scale study on the effect of team composition on multiple measures of team effectiveness. We use a dataset from the largest multiplayer online battle arena (MOBA) game, Honor of Kings, with 96 million matches involving 100 million players. We measure team effectiveness based on team performance (whether a team is going to win), team tenacity (whether a team is going to surrender), and team rapport (whether a team uses abusive language). Our results confirm the importance of team diversity and show that diversity has varying effects on team effectiveness: although diverse teams perform well and show tenacity in adversity, they are more likely to abuse when losing than less diverse teams. Our study also contributes to the situation vs. personality debate and show that abusive players tend to choose the leading role and players do not become more abusive when taking such roles. We further demonstrate the predictive power of features based on team composition in prediction experiments.

Tables

Table 1. Table 1 . Description and frequency of each role.

Role	Description	Freq.(%)
warrior	Warriors have a large pool of health and considerable damage; they often undertake the stress from enemies in side lanes.	27.1
mage	Mages have heavy spell damage, but a small pool of health; they often take the middle lane and prepare to assist teammates.	25.8
marksman	Marksmen have a small pool of heath but can cause heavy damage; they need protection and may need time to grow.	25.7
assassin	Assassins are explosive and control the pace, but the health pool is often small; they are usually the leading role in a team.	11.5
support	Supports always aid teammates; they absorb damage while disrupting opponents by stunning and displacing them.	9.9

Table 2. Table 2 . Statistics of the dataset.

Statistics	Number(fraction/SEM)
Overall statistics
$\#$ranked games	$54,434,817$
$\#$team compositions	$126$
$\#$surrender games	$5,807,005\left(10.7\%\right)$
$\#$abusing games	$30,668,577\left(56.3\%\right)$
$\#$early games($t\le$ 11 min)	$5,049,934\left(9.3\%\right)$
$\#$middle games(11 ¡ $t\le$ 16 min)	$23,089,993\left(42.4\%\right)$
$\#$late games($t$ ¿ 16 min)	$26,294,890\left(48.3\%\right)$
Top five frequent combinations
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	$16,186,003\left(14.9\%\right)$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	$14,516,443\left(13.3\%\right)$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	$12,630,805\left(11.6\%\right)$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	$9,124,177\left(8.4\%\right)$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	$8,929,680\left(8.2\%\right)$
Personal statistics
$\#$target players	$1,306,754$
$\#$target players who have played ranked games	$1,173,372$
$\#$ranked games each player has	$48.4\left(0.05\right)$
average winrate	$52.4\%\left(1.0\times {10}^{-4}\right)$
average abusing probability	$9.3\%\left(1.0\times {10}^{-4}\right)$
average surrender probability	$5.0\%\left(6.9\times {10}^{-5}\right)$
$\#$target players who have played at least 20 ranked games	$758,494\left(58.0\%\right)$

Table 3. Table 3. Example team compositions and winning rates.

Team compositions	Win rate	Used frequency(%)
team compositions with the highest winning rate
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	53.6%	0.3
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	53.2%	2.0
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	52.6%	14.9
team compositions with the lowest winning rate
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	19.7%	$3.1\times {10}^{-3}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	17.4%	$2.2\times {10}^{-2}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	8.3%	$7.0\times {10}^{-4}$
team compositions that are most used
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	52.6%	14.9
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	52.0%	13.3
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	48.5%	11.6

Table 4. Table 4. Top five role property features in winning team prediction. Feature importance is defined as the frequency of a feature being used in a tree.

rank	features	importance(%)
$\#7$	minimum winrate of $\text{<img>}-\text{<img>}$	$1.1$
$\#8$	minimum winrate of $\text{<img>}-\text{<img>}$	$1.1$
$\#9$	minimum winrate of $\text{<img>}-\text{<img>}$	$1.0$
$\#10$	std(rank level) of	$1.0$
$\#11$	average rank level of	$0.9$

Table 5. Table 5 . The list of available information for each game.

Tag	Description
Player’s basic information
$\#$grade-of-rank	Player’s rank level, 0-26.
$\#$heroid	Hero that player uses in this game.
Game’s basic information
$\#$game-time	UTC timestamp, game start time.
$\#$duration	Time duration of the game.
$\#$gameresult	Win/lose result for the player.
$\#$surrender-tag	The tag of whether the game ends with surrendering.
In-game information
$\#$kill-cnt	Number of kills.
$\#$dead-cnt	Number of deaths.
$\#$assist-cnt	Number of assistance kills.
coin-cnt	Amount of coins.
Player’s historical records
rank-mvp	Number of mvp he/she got in his/her last 10 ranked games.
$\#$ten-rank-win	The number of winning in his/her last 10 ranked games.
$\#$ten-rank-total	Total number of previous 10 ranked games.
Chat information(if exists)
$\#$timestamp	UTC timestamp, the time of one chat record.
$\#$abuse-tag	The tag of whether this chat record includes abusing words.

Table 6. Table 6. Team compositions with the highest and lowest surrender probability ( t ≤ 11 ​ m ​ i ​ n 𝑡 11 𝑚 𝑖 𝑛 t\leq 11min ).

Team compositions	Surrender prob.	Used frequency(%)
team compositions with the highest surrendering rates
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	84.6%	$3.0\times {10}^{-4}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	82.4%	$6.6\times {10}^{-4}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	79.8%	$9.7\times {10}^{-5}$
team compositions with the lowest surrendering rates
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	34.8%	$4.6\times {10}^{-4}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	34.3%	$2.1\times {10}^{-5}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	33.6%	$1.5\times {10}^{-4}$

Table 7. Table 7. Team compositions with the highest and lowest abusing probability.

Team compositions	Abusing prob.	Used frequency(%)
team compositions with the highest abusing rates
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	56.2%	$2.2\times {10}^{-3}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	55.9%	$2.6\times {10}^{-3}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	55.6%	$7.2\times {10}^{-3}$
team compositions with the lowest abusing rates
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	30.6%	$1.7\times {10}^{-3}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	29.7%	$1.5\times {10}^{-5}$
$\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}-\text{<img>}$	28.7%	$8.6\times {10}^{-5}$

Equations

$$\mathbb{E}(\text{\#abusing players in a team})=\sum _{v\in \mathtt{team}}{P}_{\mathtt{abuse}}(v)$$

Full text

team composition, team performance, tenacity, toxic behavior, MOBA

††ccs: Applied computing Law, social and behavioral sciences

1. Introduction

The increasing complexity and scale of tasks in modern society require individuals to work together as a team (Wuchty et al., 2007; Cohen and Bailey, 1997). Crucial to the effectiveness of a team are the individuals in the team, i.e., team composition. Extensive prior research has studied team roles to analyze team composition (Parker, 1990; Senior, 1997; Belbin, 2011; Partington and Harris, 1999; Stewart et al., 2005; Belbin, 2012). The most influential work is Belbin’s team role framework, also known as Belbin Team Inventory. A central hypothesis in Belbin’s framework is that balance in team roles is associated with team performance. Belbin (2012) defines nine roles 222See https://en.wikipedia.org/wiki/Team_Role_Inventories for a quick explanation., and finds that teams with certain role combinations result in poor performance, even if they are formed by members with the sharpest mind and the most experience. However, team roles are usually implicit in real life and it is difficult to identify such nine roles in most contexts.

Another challenge in studying the effect of team composition on team effectiveness arises from the definition of effectiveness. Team performance is the most straightforward definition and has been studied in a battery of studies (Salas et al., 2008; Brannick et al., 1993; Cohen and Bailey, 1997; Dirks, 1999). Example measures of team performance include the impact of published papers from a team (Wuchty et al., 2007), winning a sports game (Audas et al., 2002; Kim et al., 2017), etc. However, the effectiveness of a team can also be reflected by the tenacity in face of adversity, and the rapport between team members (Bradner et al., 2005). The effect of team composition likely depends on the definition of team effectiveness. Therefore, it is important to understand the effect of team composition on the effectiveness of teams in multiple measures.

To address these two challenges, we identify multiplayer online battle arena (MOBA) as an ideal testbed and provide a large-scale study on the effect of team compositions on multiple measures of team effectiveness. We use a dataset from Honor of Kings, the largest MOBA game in the world, and Fig. 1 illustrates the process of an example game. This platform is ideal for studying the effect of team composition for three reasons. First, there are defined roles in the game and players choose their roles to form a team. Given that there are five players in each team, we are able to enumerate all possible team compositions. We also have information about the characteristics of each role. Second, the popularity of Honor of Kings leads to digital traces of hundreds of millions of players in hundreds of millions of games. These detailed game records allow us to explore the notion of team “effectiveness” beyond the simple measure of team performance, winning or losing. For instance, we study the effect of team composition on abusive language use, which reflects the rapport in a team. This research question naturally connects to the literature on toxic behavior in online communities and gaming (Cheng et al., 2017; Kwak et al., 2015; Cheng et al., 2015).

Finally, we have access to the past history of players for capturing their background and experience. Historical information enables us to investigate questions in the “situation vs. personality” debate (Donellan et al., 2009; Kenrick and Funder, 1988): how much of our behavior is determined by fixed personality traits or by the specific situation at hand? We will explore this question in the context of toxic behavior in online gaming.

**Organization and paper highlights. ** In this paper, we conduct a large-scale study on the effect of team composition in the context of online gaming. Our dataset comes from a popular MOBA game in China, Honor of Kings, and includes 96 million games and 100 million players in total. We provide details of the dataset and character roles in §2.

We quantitatively study the effect of team composition on three distinct definitions of team effectiveness: (§3) team performance — whether a team is going to win — is directly based on the result/goal in a game and is the most common measure in prior studies; (§4) team tenacity — whether a team is going to surrender — indicates a team’s resilience to adversity and is understudied in existing literature as most studies only look at individual tenacity or perseverance (Duckworth et al., 2007; Baum and Locke, 2004); (§5) toxic behavior, i.e., abusive language use, reflects the rapport in a team and is extracted from the activities during a game.

Our results on team effectiveness confirm the theory that balance in team roles is important for effective teams. We find that diverse teams are more likely to win. Furthermore, diverse teams are less likely to surrender, indicating a higher level of tenacity. Note that the likelihood to surrender does not correlate with winning rate for commonly used team compositions. However, we observe diverging effects on abusive language use: diverse teams are more likely to abuse when losing, but are less likely to abuse when winning.

In addition to exploring the effect of team composition on team effectiveness, we investigate individual-level abusive language use to understand how teamwork influences individuals and shed light on the “situation vs. personality” debate (Donellan et al., 2009; Kenrick and Funder, 1988). We find that players who choose assassins (the leading role in a team) are more likely to abuse. We further demonstrate that the reason is that abusive players are more likely to choose assassins, instead of the alternative explanation that a player becomes more abusive when playing assassins. In a similar vein, we find that experienced players are more likely to abuse, but not when their teammates are also experienced. A design implication is that matching players with similarly experienced players can improve the rapport in a team.

To demonstrate the effectiveness of modeling team composition, we formulate predictions tasks for all three measures of team effectiveness in §6. We show that features based on team compositions can consistently outperform individual player information. In fact, modeling single roles and two-role combinations provides most of the predictive power. Finally, we present related work in §7 and offer our concluding thoughts in §8.

2. Dataset

Our dataset is provided by Tencent333Honor of Kings is developed and published by Tencent, which is one of the largest Internet companies and also the largest game service provider in China. and comes from a multiplayer online battle arena (MOBA) game: Honor of Kings, the most profitable game in the world444See https://en.wikipedia.org/wiki/Wangzhe_Rongyao for an introduction of the native version of Honor of Kings (Wangzhe Rongyao).. Similar to many other MOBA games such as Dota2 and League of Legends, Honor of Kings involves two teams to battle with each other. Each team consists of five players and the success of a team depends on the chemistry and collaboration between the five team members. The main motivation of our study is that players are expected to take certain roles when they select characters. These roles are designed to complement each other and “good” teams usually require particular compositions of roles. In this paper, we aim to explore the effect of team composition on different measures of team effectiveness.

There are multiple types of games in Honor of Kings. Ranked games are used to estimate a player’s rank level in the game; while the other games are for practice or for fun. Since players care much less about unranked games than ranked games, all of our analyses are based on ranked games. Unranked games are only used to analyze players’ role preferences.

**Dataset description. ** Our dataset is derived from daily logs spanning three weeks in the August of 2017. Specifically, we first randomly sample 20K games in each day of the third week, thereby obtaining 140K games and 1.3M players participating in these games. Among them, there are 78K ranked games. Our prediction experiments in §6 are based on these ranked games, which we refer to as prediction games. We then retrieve all the games involving any of these 1.3M players in the first two weeks to obtain player information for prediction. In total, we obtain 95.6M games and 100.2M players. Among these games, there are 54.4M ranked games on which most analyses in §3, §4, and §5 are based, as under those analytical settings, we do not need historical information of every player in a team. There are several exceptions where we need historical statistics of every team member, so we use prediction games in those cases and will point out in the text.

The log of each game includes its start time, end time, result (which team wins), gaming stats recorded when the game ends, and text messages in the chat room with abusive labels. In addition, we have access to basic player information beyond our dataset, e.g., number of winning games, number of games played as MVP (Most Valuable Player) and total number of games in the past ten ranked games, etc. See Table 5 in §A for details.

**Team roles. ** There are five roles in Honor of Kings: warrior ( 

 ), mage ( 

 ), marksman ( 

 ), assassin ( 

 ), and support ( 

 ). A team is composed of different roles depending on each player’s choice. Henceforth, we use five icons to represent a team composition. For instance, (\,\vbox{\hbox{\mathchoice{\includegraphics[height=12.91663pt]{figure/warrior.pdf}}{\includegraphics[height=12.91663pt]{figure/warrior.pdf}}{\includegraphics[height=10.76385pt]{figure/warrior.pdf}}{\includegraphics[height=8.61108pt]{figure/warrior.pdf}}}}\,,\,\vbox{\hbox{\mathchoice{\includegraphics[height=12.91663pt]{figure/mage.pdf}}{\includegraphics[height=12.91663pt]{figure/mage.pdf}}{\includegraphics[height=10.76385pt]{figure/mage.pdf}}{\includegraphics[height=8.61108pt]{figure/mage.pdf}}}}\,,\,\vbox{\hbox{\mathchoice{\includegraphics[height=12.91663pt]{figure/mage.pdf}}{\includegraphics[height=12.91663pt]{figure/mage.pdf}}{\includegraphics[height=10.76385pt]{figure/mage.pdf}}{\includegraphics[height=8.61108pt]{figure/mage.pdf}}}}\,,\,\vbox{\hbox{\mathchoice{\includegraphics[height=12.91663pt]{figure/assassin.pdf}}{\includegraphics[height=12.91663pt]{figure/assassin.pdf}}{\includegraphics[height=10.76385pt]{figure/assassin.pdf}}{\includegraphics[height=8.61108pt]{figure/assassin.pdf}}}}\,,\,\vbox{\hbox{\mathchoice{\includegraphics[height=12.91663pt]{figure/assassin.pdf}}{\includegraphics[height=12.91663pt]{figure/assassin.pdf}}{\includegraphics[height=10.76385pt]{figure/assassin.pdf}}{\includegraphics[height=8.61108pt]{figure/assassin.pdf}}}}\,) stands for a team with one warrior, two mages, two assassins, and no marksman or support. In total, there exist 126 team compositions. Table 1 presents the characteristics and frequency of each role555The boundary between roles can sometimes be blurry, especially considering different player tactics.. Figure 2 shows the frequency of two roles co-occurring in the same team and Table 3 shows several role combinations with its winning rate and frequency. We can clearly see that team compositions are not used equally by players: single roles and pairs of roles both vary greatly in frequency. Please refer Table 2 for detailed statistics of our dataset.

3. Team Composition and Winning

We first investigate the effect of team composition on whether a team wins or not, a direct measure of team performance. Although the matching system in online gaming is designed to balance the ability of two teams, we observe that the winning rate of most team compositions is significantly lower than 50%, indicating that some role combinations cannot effectively work with each other.

**Although two teams are with similar “skills”, there exist winning and losing team compositions (Fig. 3(a)). **Since we have 126 different team compositions in total, it is straightforward to estimate the winning rate of each team composition using the fraction of winning games, $\frac{\#\text{winning~{}games}}{\#\text{games}}$. Fig. 3(a) shows that the winning rate of different team compositions spans a wide range, from 8.3% to 53.6%. It seems that some team compositions are doomed to lose. In addition, the figure also shows the cumulative distribution of #teams for each team composition, which suggests that most losing teams do not occur frequently. Notice that a similar plot will also be made for surrendering and abusing and we will see that the CDFs present very different shapes.

The fluctuation of winning rate between different team compositions among all games may not due to the effect of team compositions; another possible explanation is the ability of individual players: only relatively bad players choose certain bad team compositions. Although a matching system is designed to balance the ability of the players in two teams, it is difficult to always make sure that two teams exactly have the same ability. Fig. 3(b) shows the distribution of the rank level gap ($\delta$) between two teams666 Rank level indicates a player’s gaming skill and ranges from 0 to 26. . To calculate $\delta$, we sum up the individual rank level in a team, and subtract the result of losing team from that of winning team. By the effect of the matching system, the rank gaps in over $90\%$ games are relatively small (within $\pm 5$). These minor gaps, however, are still associated with a significant difference in winning rate: only $28.7\%$ games end up with the winning of the lower ranked team. Therefore, to demonstrate the influence of team compositions on winning rates and to exclude the ability factor, we control the rank gap between teams in Fig. 3(a): we focus on games with $\delta=0$ (red plots) and $\delta\leq 4$ (gray plots) respectively.

To further illustrate winning and losing team compositions, we present team compositions with the highest (lowest) winning rate in Table 3777 We make similar tables as Table 3 for surrendering and abusing. See §A for reference. (controlling the rank gap $\delta=0$). Upon a careful examination, these losing teams do not often occur, reflected also by the cumulative distribution curve in Fig. 3(a). In total, the bottom $90$ losing team compositions only take $11.1\%$ of all teams. This suggests that most players choose a reasonable team composition. However, the last three rows in Table 3 show that the most common team compositions still vary in winning rates, from 48.5% to 52.6%.

**Diverse teams tend to perform well (Fig. 3(c), 3(d)). ** An observation that stands out in Table 3 is that teams with the highest winning rate consist of more role categories than teams with the lowest winning rate. For instance, support ( 

 ) or assassin( 

 ) dominates the two compositions with the lowest winning rate, while all the top winning teams have at least four roles.

We further explore this observation by examining how winning rate changes as the number of roles in a team grows (Fig. 3(c)). We find that diverse team compositions are much more likely to win. One explanation is that these roles are designed to complement each other and teams with too few roles have weaknesses to be exploited. For instance, a team with no mage ( 

 ) or marksman ( 

 ) cannot cause sufficient damage in a game. Fig. 3(d) shows that most teams are employing at least three role categories, which is consistent with the observation in Fig. 3(a) that the losing team compositions on the left are not usually used.

4. Team Composition and Surrendering

As nothing worth achieving is going to be easy, tenacity is an important characteristic of an effective team. In the context of Honor of Kings, team tenacity can be reflected by not surrendering easily. The surrendering procedure is as follows: a player can propose to surrender and if at least four of five team members agree, the team will surrender as a whole and lose immediately. In this section, we examine the effect of team composition on team tenacity.

**Surrender probability varies across team compositions in early games (Fig. 4(a), 4(b)). ** Similar to winning, surrender happens at the end of a game. But surrender requires extra careful analysis because surrender may not always reflect team tenacity in Honor of Kings and other MOBA games. For example, when the opponents are destroying the team base and all team members are killed, it is time to move on and surrender is simply a sign of “game over”. This is common in late games. Therefore, it is necessary to distinguish surrender in different game stages.

We examine the mean and the standard deviation of surrender probability across team compositions conditioned on game duration in Fig. 4(a). We make two observations. First, the surrender probability monotonically decreases as game duration increases, partly because a team is only allowed to surrender starting from the 6th minute and games end in 6-8 minutes mostly because a team surrenders. Second, the standard deviation across team compositions first increases and then decreases as game duration increases, indicating that in late games every team surrenders with similar probability to signal game over. Based on Fig. 4(a), we define three game stages: early games ($t\leq 11$), where the surrender probability has great mean and great std; late games ($t>16$), where the surrender probability has small mean and small std; middle games ($11<t\leq 16$), the middle part888A gaming related reason for this split is that 11-minutes marks a watershed because the “Overlord” and “Dark Tyrant” (two important non-player characters) appear at the 10${}^{\mbox{th}}$ minute and game state can change significantly from 11 to 16 minutes when the characters get killed.. Proportions of games in each stages are shown in Table 2.

Having established the three types of games, we hypothesize that surrender in early games is the most indicative of team tenacity and varies across team compositions, while surrender in middle and late games is more of a formality and should not depend on team compositions. To study that, we sort team compositions by surrender probability in early games in Fig. 4(b). According to the same order, we show the surrender probability in middle games and late games, as well as the cumulative distribution function of #teams. Consistent with our hypothesis, the surrender probability of different team compositions span a wide range from 33.6% to 84.6% in early games, but is pretty stable in middle games and late games. The CDF of #teams presents a different shape from that in Fig. 3(a): commonly used team compositions are neither the most tenacious nor the least tenacious. Table 6 shows the most tenacious and the least tenacious team compositions.

**Diverse teams tend to be tenacious (Fig. 4(c)). ** Role diversity influences surrender probability in early games, especially when the number of roles is large. It seems that diverse teams with five roles are the most tenacious in early games. In comparison, the influence on middle and late games is minimal.

**A weak team can still be tenacious (Fig. 5). ** We have shown that diverse teams tend to both perform well and show tenacity in adversity. One concern is that these two measures are correlated and tenacity is simply a side effect of team strength (winning). This hypothesis suggests that winning team compositions should be less likely to surrender. To further understand this issue, we study the correlation between surrender probability and winning rate for different team compositions. Fig. 5(a) shows that there is indeed a negative correlation between winning rate and surrender probability in early games. There is little correlation in middle and late games.999In fact, there is a small significantly positive coefficient in late games, which is another evidence that strength does not entail tenacity. However, it seems that the correlation between winning rate and surrender probability in early games is dominated by a few outliers with very low winning rates. Given that we know that these teams only take a small fraction of all games in §3, we filter infrequent team compositions that appear less than 10,000 times and find that in the remaining team compositions ($98.7\%$ of all teams), surrender probability is not correlated with winning rate in all game stages (Fig. 5(b)), suggesting that team tenacity is almost independent of team strength/performance.

5. Team Composition and Abusive Language Use

Our final measure of team effectiveness is concerned with the rapport in a team during the game. We use abusive language use to capture the rapport. Toxic behavior in online communities and gaming has received significant interests from our research community recently (Cheng et al., 2017, 2015; Kwak et al., 2015). Here we provide the first systematic study on the effect of team composition on team-level abusive language use.

5.1. Team-level Abusive Language Use

**Team compositions vary in abusing probability (Fig. 6(a)). ** In our dataset, we have a label of whether a message uses abusive language for all text messages based on a dictionary-based method officially used by Tencent.101010Players may abuse using voice messages; here we only consider text messages. A team abuses if any player in that team abuses. For each team composition, we define abusing probability as the fraction of games that this team composition abuses.

We find that similar to winning and surrendering, team compositions vary in abusing probability, ranging from 28.7% to 56.2%. The cumulative distribution function of #teams looks much more similar to surrendering in Fig. 4(b) than winning in Fig. 3(a): most commonly used team compositions are neither the most nor the least abusive. Table 7 shows team compositions with the highest and lowest abusing probability. It is striking that all the top teams have multiple assassins ( 

 ) and all the bottom teams have multiple supports ( 

 ). We will investigate further the interaction of individual-level abusing and team-level abusing in §5.2.

**Losing teams are more likely to abuse (Fig. 6(b)). ** We hypothesize that losing teams are more likely to abuse because winning usually brings positive team morale, while losing leads to frustration and dissatisfaction (Kwak et al., 2015). This is indeed the case as shown in Fig. 6(b). Therefore, it is important to distinguish abusing probability between winning teams and losing teams.

**Diverse teams tend to abuse more when losing and abuse less when winning (Fig. 6(c)). ** We further explore the effect of role diversity on team-level abusing. We observe different trends in winning teams and losing teams. When a team wins, team diversity is associated with low abusing probability; but it becomes the other way around if a team loses.

5.2. Individual-level Abusive Language Use

Different from winning and surrendering, abusive language use is an individual behavior. It provides a great opportunity to understand the effect of team composition on individual behavior and shed light on the “situation vs. personality” debate (Donellan et al., 2009; Kenrick and Funder, 1988). Note that we conduct this part of experiments on players who have played at least 20 games to ensure sufficient samples for statistics.

**A team does not equal the sum of individuals (Fig. 7(a)). ** We define individual abusing probability based on the fraction of games that a player abuses in.111111All individual abusing probability is based on individual level samples. Individual samples can be grouped based on criteria other than all games played by a single player, e.g., we compute individual abusing probability for all winning (losing) games of a player in Fig. 7(a) and for all players who choose a particular role in Fig. 7(b).

Because of linearity of expectation, the total number of expected abusing players in a team is simply the sum of each player’s abusing probability:

[TABLE]

where $P_{\mathtt{abuse}}$ is estimated separately when a player wins or loses because abusing is associated with losing. This expectation is only valid if all players act independently in a team, but studies on teamwork have shown that a team does not equal the sum of individuals (Belbin, 2012). Therefore, we examine the discrepancy between observed values and expected values to understand how team composition influences individual players. This analysis is done on prediction games since it requires historical information of every team member.

Figure 7(a) shows the difference between observed and expected values for winning and losing teams, where win/lose diff. refers to the disparity between observed and expected number of abusing players($\mathbb{O}(\text{\#abusing players})-\mathbb{E}(\text{\#abusing players})$). For most team compositions, the difference between observed values and expected values is not zero, indicating that individuals abuse differently depending on team compositions. In particular, in the most commonly used teams (when the CDF grows quickly on the right of the plot), individuals are more likely to abuse than expected when losing.

**Players who prefer leading roles are more abusive (Fig. 7(b), 7(c), 7(d)). ** There always exists a leader or a major contributor in a team. In Honor of Kings, assassins usually take this role in a team because of their explosiveness: assassins can carry the team and control game pace. A failed assassin may lead the team to lose.

Fig. 7(b) shows that assassins are more likely to abuse than other roles. We call this the “abusive assassin” phenomenon. One natural question arises: are assassins more abusive because abusive players tend to choose assassins, or players become more abusive when choosing assassins?

To answer this question, we compare the abusing probability of the same roles chosen by different players and the same player choosing different roles. We define the experienced role for a player if that player chooses a role frequently, i.e., playing in more than $50$% of games. This procedure identifies experienced assassin players, experienced warrior players, etc. Fig. 7(c) presents the abusing probability of experienced assassin players when they choose each role (the same player choosing different roles). We observe that these experienced assassin players are much more likely to abuse than other players no matter what role they choose.

Fig. 7(d) examines the alternative hypothesis: players become more abusive when choosing assassins. We compare the same player’s abusing probability when they choose assassins with choosing other roles, grouped by their experienced roles. We find that players do not become more abusive when choosing assassins. If anything, experienced mages and experienced marksmen are actually more abusive when they choose roles other than assassins.

Overall, our results in Honor of Kings support the hypothesis that abusive players tend to choose assassins, the leading role in a team, and players do not become abusive when choosing assassins.

**Experienced players are more likely to abuse, but not when they play with other experienced players (Fig. 8). ** Studies have shown that experienced individuals tend to abuse novices (Honeycutt, 2005). We hypothesize that a player is more likely to abuse when playing his experienced role. In addition to using 50% as a threshold to define experienced roles, we also use 70% and 60% here. Fig. 8(a) shows that players are more abusive when playing their experienced roles no matter what threshold is.

We next further explore how players are influenced by teammates when choosing experienced roles. Fig. 8(b) shows that as the number of experienced roles increases among a player’s teammates, his individual abusing probability player decreases (since this analysis requires historical information of every team member, it is done on prediction games). This observation indicates that a player becomes more likely to abuse when choosing experienced roles partly because other less experienced players do not meet this experienced player’s expectation and cause frustration.

6. Prediction Experiments

In this section, we set up three prediction tasks for winning, surrendering, and abusing respectively to further examine the effect of team composition in a predictive setting. Overall, we find that features based on team composition consistently outperform features based on individual players.

6.1. Experiment Setup

**Prediction tasks and evaluation metrics. ** Our goal is to evaluate whether modeling team composition brings extra predictive power to modeling individual players. Because team composition cannot be altered after a game starts, we focus on making predictions before a game starts and consider the following three prediction tasks. Our results can potentially inform both players in choosing roles and the matching-making system to match players.

• •

Winning team prediction(Xia et al., 2017). This task aims to predict which team will win before the game starts. We randomly swap the order of two teams and label a game positive if the first team wins and negative if the second team wins. This leads to a balanced dataset, so we use accuracy for evaluation.

• •

Surrender prediction. This task aims to predict whether the losing team surrenders in a game. Only 8K games (10.5%) end with surrender, so we use F1 as evaluation metric in this task.

• •

Abuse prediction. Before a game starts, this task predicts whether abusive language use happens in the game. Abusive language use happens in 44K games (57.93%), so we use accuracy for evaluation in this task.

**Feature sets. ** We consider the following feature sets.

• •

Baseline. People’s mood varies over time and is associated with abusive language use (Cheng et al., 2017). Our baseline features include day of the week and hour of the day encoded as one-hot vectors.

• •

Player information. Depending on the task, we compute each player’s rank level, winning rates in the last 10 ranked games, surrender probability, and abusing probability from historical games. We use the average, max, min, and standard deviation of those properties of all players in a team as features.

• •

Role properties. A role combination of $k$ refers to all possible combinations of $k$ roles. For instance, one-role combinations refer to the five roles in Table 1, two-role combinations refer to 15 possible combinations of two roles ($\binom{5}{1}+\binom{5}{2}$). We further incorporate player information in role combinations of both teams. For each role combination in a team, we compute the average of role-based player information in an instance of that combination and then use the average, max, min, and std of all instances in that combination as features. Taking an example to further explain role-based player information, the feature winning rate of role $c$ for player $v$ is defined as the probability of $v$ wins when she plays $c$. We only consider up to two-role combinations ($k\leq 2$) when comparing prediction performances. Later we will explore the effect of considering three-role combinations up to five-role combinations, i.e., full team compositions.

We also use the union of all feature sets (“total”). Because we find that game duration plays an important role in surrender, we add game duration to every feature set for surrender prediction. In abuse prediction, we do not distinguish two teams.

**Prediction setup. ** For each task, we conduct 5-fold nested cross-validations on prediction games in the third week and player information is extracted from historical games in the first two weeks. We use Xgboost (Chen and Guestrin, 2016) since it is often the winning solution in Kaggle competitions and shows strong performance compared to other methods such as logistic regression in our preliminary experiments. We grid search hyperparameters in max depth ({1, 3, 5, 7, 9}), learning rate ({0.1, 0.2}), and number of estimators ({100, 200, 300}).

6.2. Experiment Results

**Prediction performance (Fig. 9(a), 9(b), 9(c), and 9(d)). ** In all prediction tasks, we observe a consistent pattern that $\textit{total}>\textit{role property}$ $>\textit{player information}>\textit{baseline}$. Player information achieves relatively better performance than baselines since individual characteristics definitely influence team effectiveness in some degree. By incorporating player information with role combinations, role properties consistently outperform player information. The extra predictive power from modeling team composition can be reflected by the difference between total and player information: e.g., 70.4% vs 67.7% in winning prediction for all games, and 67.8% vs 65.0% in abusing prediction. In winning prediction, the effect of team composition is further amplified when the rank gap being controlled: 56.3% vs 53.3%. The difference is the least prominent in the prediction of surrender, as this behavior is often highly influenced by teammates and various situations during games, while role properties still improve the performance by 1.5% in surrender prediction.

Recall that Fig. 3(b) shows a clear influence of rank level on winning rate. Therefore, to further understand the effect of team compositions, we report the performance on games where the rank level gap $\delta=0$ in Fig. 9(b). We find that after the rank level gap is controlled, this task becomes much more challenging and the performance of all feature sets drop significantly compared to Fig. 9(a). However, the difference between role properties and player information almost doubles (1.6% vs. 0.9%), demonstrating the effectiveness of incorporating team compositions, especially when the two teams are similarly skilled. In contrast, rank level does not affect the performance when predicting surrender or abuse.

**Modeling two-role combinations is sufficient to capture the effect of team composition (Fig. 10). ** To further illustrate the effect of team compositions, we examine the performance of adding $k$-role combinations to player information and baseline one by one: we first add single-role features, then add features based on two-role combinations, until adding features based on entire team compositions. Taking winning team (on all games) and abuse prediction as examples, Fig. 10 shows that the performance improvement saturates after adding two-role combinations. It suggests that modeling single roles and two-role combinations provides most of the predictive power.

**Feature analysis. ** We finally explore the importance of features by using winning team prediction task as an example. When combining role properties and player information, average rank levels of two teams are the most important features (ranked top-2), as well as average individual winning probability (in top-5). To understand the key features in role properties, we show the top five features in role properties in Table 4. Consistent with the characteristics of roles where mage ( 

 ), marksman ( 

 ) and warrior ( 

 ) are most commonly used, the combinations of these roles are more important in role properties. And as marksman ( 

 ) and mage ( 

 ) are the key roles to cause damages, role combinations related with these roles are particularly significant. The first three of the top five are all two-role features, which again indicates that modeling two-role combinations is effective to capture the team performance.

7. Related Work

In addition to studies mentioned throughout the paper, we discuss additional related work in three strands.

**Team formation. ** Researchers from the data mining community have formulated the problem of team formulation as a constrained optimization problem based on each individual’s skills and their social networks (Anagnostopoulos et al., 2010, 2012; Li et al., 2017; Lappas et al., 2009). For instance, Anagnostopoulos et al. (2012) take into account of the task requirements, individual workloads, and team communication costs. Li et al. (2017) study a family of problems in team enhancement including team member replacement, team expansion, and team shrinkage.

**Social/team roles. ** Digitalization of human traces have increasingly made implicit social/team roles explicit and enabled large-scale study on theories of social/team roles (Belbin, 2011, 2012; Biddle, 1986; Yang et al., 2016). Yang et al. (2015) study how social roles influence the process of online information diffusion. Danescu-Niculescu-Mizil et al. (2012) demonstrate changes in language use after a Wikipedia user becomes an administrator. Yang et al. (2016) infer editor roles on Wikipedia based on behavior traces. In the context of gaming, Stetina et al. (2011) examined problematic gaming behavior and depressive tendencies among people who play different types of online-games.

**Online gaming. ** Many researchers have recognized that online gaming, especially MOBA and MMOGs (Massively Multiplayer Online Games), can serve as a platform for studying individual and team behavior, group norms, and communities (Ducheneaut et al., 2007; Kim et al., 2017). In such games, players not only develop individual skills, but also coordinate and communicate with others. The existence, significance, and influence of such social interactions have been examined in many studies  (Ducheneaut et al., 2006; Hudson and Cairns, 2014; Losup et al., 2014; Freeman, 2018; Szell and Thurner, 2010).

Team performance is the most heavily studied topic among our three measures of effectiveness. Pobiedina et al. (2013a, b) explore factors that influence player’s performance in Dota 2 and reveal that team-level interactions, especially the number of friends in a team, relate to team performance. Similar to our work, prior studies such as (Agarwala and Pearce, 2014; Conley and Perry, 2013) demonstrate that different role combinations may greatly influence team performance. Deep learning based recommendation systems are also built to recommend heroes or roles to players(Sapienza et al., 2018).

Furthermore, gamification can also potentially play an important role in education and scientific discovery (Khatib et al., 2011). In particular, Petter (2017) suggests that researchers should embrace the growing popularity of online gaming and seek opportunities in studying online gaming in the context of education, individual, and team behavior. In addition to these exciting opportunities, online gaming leads to issues such as addiction and depression (Stetina et al., 2011; Kuss et al., 2012).

8. Conclusion

In this paper, we study the effect of team composition in the largest multiplayer online battle arena (MOBA) game: Honor of Kings. We quantitatively show the varying effects of team composition on team performance (winning), team tenacity (surrender), and team rapport (abusive language use): although diverse teams tend to perform well and show tenacity in adversity, they are more likely to abuse when losing. The double-edged influence of team diversity suggests the importance of balancing team composition. We also examine how team composition influences individual behavior in abusive language use. In addition to showing that a team is not the sum of independent individuals, we contribute to the “situation vs. personality” debate and show that assassins abuse more because abusive players tend to choose assassins instead of players becoming abusive when choosing assassins. Our work suggests that the gaming environment may be improved by adjusting team matching and preventing players from using team compositions that correlate with increased abusive language use.

**Limitations. ** Although we control for roles in the analyses and further validate our results through prediction experiments, our study is observational and can be strengthened through experimental studies. We would like to highlight that experimental studies on team compositions are non-trivial because of the great number of possible combinations. Our work is also limited by the data that we have access to. Due to privacy issues, we do not have sensitive individual information like location and consumption record. Moreover, although Honor of Kings is the largest MOBA game, the selection bias in our data may limit the generalizability of our findings.

**Future directions. ** Several promising directions arise from our work. Our prediction experiments show that it is important to incorporate individual player information with role combinations. It remains challenging to develop a holistic model that captures the interaction between individual members, learns novel representations for role combinations, and makes even more accurate predictions. Furthermore, although MOBA games provide an ideal environment for understanding the effect of team composition, it is important to validate our findings in other scenarios, e.g., with implicit roles beyond gaming.

Appendix A Appendix

A.1. Game information

Table 5 shows the detailed information contained in the game records.

A.2. Detailed Statistics

We present team compositions with the highest (lowest) surrendering probability in Table 6, and those with the highest (lowest) abusing probability in Table 7 .