Beyond Compliance: How AI Could Help Creative Writers by Refusing Them

TL;DR

This paper explores how AI refusals can foster reflection among creative writers, examining nuanced reactions and implications for frictional AI design to promote balanced AI reliance.

Contribution

It provides empirical insights into how AI refusals can serve as strategic friction to encourage reflection in creative writing contexts.

Findings

Refusals' effectiveness varies with user preferences and situational factors.

Heterogeneous preferences influence reactions to AI refusals.

Design implications include integrating different levels of AI compliance.

Abstract

Mainstream creativity support design prioritizes compliant AI for seamless writing interactions, but concerns over inappropriate AI reliance highlight the need for designs fostering reflection on balanced AI and non-AI resource use. Theoretically, intentional AI non-compliance, refusals (saying ``no'' to requests), could introduce such reflection through friction stronger than other bypass-able solutions. Practically, refusal content/language characteristics lead to nuanced reactions. However, little research empirically focuses on nuances beyond mandatory ethical/technical constraints, on turning refusals into strategic friction for `innocuous' requests. We address this through a qualitative study with 22 creative writers, exploring reactions to refusals to common requests across writing stages (planning, translating, reviewing). Findings suggest that reflective potential depends on heterogeneous preference alignment along situational (e.g., convergent/divergent thinking phases), cognitive (e.g., domain beliefs), and relational (e.g., AI roles) dimensions. We discuss implications for creativity support, broader issues (e.g., AI addiction), and frictional/seamful AI design (e.g., integrating different compliance levels).