CrossTalk: Intelligent Substrates for Language-Oriented Interaction in Video-Based Communication and Collaboration

TL;DR

CrossTalk introduces intelligent substrates into videoconferencing systems to recognize user intentions and provide unobtrusive assistance, aiming to enhance communication fluidity and collaboration effectiveness.

Contribution

The paper presents a novel system, CrossTalk, integrating language-based intent recognition and lightweight interactions into video communication platforms.

Findings

Enables more fluid and flexible communication

Improves collaboration through proactive assistance

Validates design concepts through system implementation

Abstract

Despite the advances and ubiquity of digital communication media such as videoconferencing and virtual reality, they remain oblivious to the rich intentions expressed by users. Beyond transmitting audio, videos, and messages, we envision digital communication media as proactive facilitators that can provide unobtrusive assistance to enhance communication and collaboration. Informed by the results of a formative study, we propose three key design concepts to explore the systematic integration of intelligence into communication and collaboration, including the panel substrate, language-based intent recognition, and lightweight interaction techniques. We developed CrossTalk, a videoconferencing system that instantiates these concepts, which was found to enable a more fluid and flexible communication and collaboration experience.