MagicCopy: Bring my data along with me beyond boundaries of apps

TL;DR

MagicCopy is an AI-powered tool that simplifies cross-application data transfer by automatically extracting, transforming, and formatting data based on user instructions, reducing manual effort.

Contribution

It introduces MagicCopy, a novel AI-based system that enables seamless, context-aware data transfer across different applications using natural language instructions.

Findings

Participants quickly learned to use MagicCopy across multiple app pairs.

Users explored diverse applications, enhancing cross-application workflows.

MagicCopy reduced manual data reformatting efforts.

Abstract

People working with data often move their data across multiple applications, because they rely on these apps' complementing user experiences to best complete their tasks. Since traditional copy-and-paste approaches do not accommodate diverse table representations adopted by different apps, users spend considerable effort to reconstruct data formats and visual representations, making cross-app workflows costly. For example, when transferring a spreadsheet table with conditional formatting to a markup document, users spend substantial time translating its structure into appropriate tags and manually reformat color. This paper introduces MagicCopy, an AI-powered cross-app copy-and-paste, leveraging source and target contexts and user-specified instructions in natural language to automatically extract, parse, transform, and (re)format data from one app to another. In a study with sixteen participants, users quickly learned and applied MagicCopy to move data across three pairs of tools. Participants further explored diverse applications of MagicCopy to support more streamlined crossed-application interaction in their workflows.