Beyond the Chat: Executable and Verifiable Text-Editing with LLMs

TL;DR

InkSync enhances document editing with LLMs by providing executable, transparent, and verifiable edits, addressing factual accuracy issues through a multi-stage verification process, and improving user experience over standard chat interfaces.

Contribution

This paper introduces InkSync, a novel editing interface that integrates executable suggestions and a three-stage fact verification process for more reliable LLM-assisted editing.

Findings

Usability studies show improved accuracy and efficiency.

Users prefer InkSync over standard chat interfaces.

Enhanced transparency and verifiability in document editing.

Abstract

Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing. However, standard chat-based conversational interfaces do not support transparency and verifiability of the editing changes that they suggest. To give the author more agency when editing with an LLM, we present InkSync, an editing interface that suggests executable edits directly within the document being edited. Because LLMs are known to introduce factual errors, Inksync also supports a 3-stage approach to mitigate this risk: Warn authors when a suggested edit introduces new information, help authors Verify the new information's accuracy through external search, and allow an auditor to perform an a-posteriori verification by Auditing the document via a trace of all auto-generated content. Two usability studies confirm the effectiveness of InkSync's components when compared to standard LLM-based chat interfaces, leading to more accurate, more efficient editing, and improved user experience.