DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models
Damien Masson, Sylvain Malacria, G\'ery Casiez, Daniel Vogel

TL;DR
DirectGPT introduces a direct manipulation interface for large language models, enabling more efficient and intuitive interactions through continuous representations, manipulable outputs, and undo features, demonstrated with improved user performance.
Contribution
This work presents a novel interface layer that applies direct manipulation principles to LLM interactions, enhancing usability and efficiency over standard prompt-based methods.
Findings
Participants were 50% faster in editing tasks.
Users relied on 50% fewer prompts.
Prompt length was reduced by 72%.
Abstract
We characterize and demonstrate how the principles of direct manipulation can improve interaction with large language models. This includes: continuous representation of generated objects of interest; reuse of prompt syntax in a toolbar of commands; manipulable outputs to compose or control the effect of prompts; and undo mechanisms. This idea is exemplified in DirectGPT, a user interface layer on top of ChatGPT that works by transforming direct manipulation actions to engineered prompts. A study shows participants were 50% faster and relied on 50% fewer and 72% shorter prompts to edit text, code, and vector images compared to baseline ChatGPT. Our work contributes a validated approach to integrate LLMs into traditional software using direct manipulation. Data, code, and demo available at https://osf.io/3wt6s.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Text Readability and Simplification
