Comparing Native and Non-native English Speakers' Behaviors in Collaborative Writing through Visual Analytics

TL;DR

This paper introduces extsc{COALA}, a visual analytics tool designed to compare native and non-native English speakers' behaviors in collaborative writing, addressing data complexity and model uncertainty.

Contribution

The paper presents extsc{COALA}, a novel visual analytics system that enhances interpretability and analysis of collaborative writing behaviors involving native and non-native speakers.

Findings

extsc{COALA} effectively visualizes behavioral differences.

User studies show improved understanding of collaboration dynamics.

Insights inform future AI tools for collaborative writing.

Abstract

Understanding collaborative writing dynamics between native speakers (NS) and non-native speakers (NNS) is critical for enhancing collaboration quality and team inclusivity. In this paper, we partnered with communication researchers to develop visual analytics solutions for comparing NS and NNS behaviors in 162 writing sessions across 27 teams. The primary challenges in analyzing writing behaviors are data complexity and the uncertainties introduced by automated methods. In response, we present \textsc{COALA}, a novel visual analytics tool that improves model interpretability by displaying uncertainties in author clusters, generating behavior summaries using large language models, and visualizing writing-related actions at multiple granularities. We validated the effectiveness of \textsc{COALA} through user studies with domain experts (N=2+2) and researchers with relevant experience (N=8). We present the insights discovered by participants using \textsc{COALA}, suggest features for future AI-assisted collaborative writing tools, and discuss the broader implications for analyzing collaborative processes beyond writing.