More Human or More AI? Visualizing Human-AI Collaboration Disclosures in Journalistic News Production

TL;DR

This study develops and evaluates visual disclosure methods for human-AI collaboration in journalism, revealing how different visualizations influence perceptions of AI involvement and highlighting the importance of nuanced disclosures.

Contribution

The paper introduces new visualization prototypes for disclosing human-AI collaboration in journalism and evaluates their effectiveness through user studies.

Findings

Chatbot visualization provides the most detailed AI collaboration information

Role-based timelines increase perceived AI contribution in human-led articles

Task-based timelines emphasize human involvement in AI-led articles

Abstract

Within journalistic editorial processes, disclosing AI usage is currently limited to simplistic labels, which misses the nuance of how humans and AI collaborated on a news article. Through co-design sessions (N=10), we elicited 69 disclosure designs and implemented four prototypes that visually disclose human-AI collaboration in journalism. We then ran a within-subjects lab study (N=32) to examine how disclosure visualizations (Textual, Role-based Timeline, Task-based Timeline, Chatbot) and collaboration ratios (Primarily Human vs. Primarily AI) influenced visualization perceptions, gaze patterns, and post-experience responses. We found that textual disclosures were least effective in communicating human-AI collaboration, whereas Chatbot offered the most in-depth information. Furthermore, while role-based timelines amplified AI contribution in primarily human articles, task-based timeline shifted perceptions toward human involvement in primarily AI articles. We contribute Human-AI collaboration disclosure visualizations and their evaluation, and cautionary considerations on how visualizations can alter perceptions of AI's actual role during news article creation.