The Harmony Index: a Utilitarian Metric for Measuring Effectiveness in Mixed-Skill Teams

TL;DR

The paper introduces the Harmony Index, a new utilitarian metric designed to quantitatively evaluate the effectiveness of mixed-skill teams, including human-agent and human-robot teams, using a large real-world dataset.

Contribution

It proposes a novel effectiveness metric, the Harmony Index, which classifies team members into sub-types based on their impact on team performance.

Findings

Validated using over 1 million interactions

Effectively distinguishes team member contributions

Potential applications in team science and remote work

Abstract

As teamwork becomes ever-more important in a new age of remote work, it is critical to develop metrics to quantitatively evaluate how effective teams are. This is especially true with mixed-modality teams, such as those that include a human and an agent or human and robot. We propose a novel utilitarian metric, the Harmony Index, which quantifies the effectiveness of team members by classifying them into four sub-types based on the result of their teaming on overall effectiveness. This index is evaluated using a real-world dataset of over 1 million interactions, and potential future uses of this index are explored in the realm of team science.