Towards a criteria-based approach to selecting human-AI interaction mode

TL;DR

This paper proposes a criteria-based framework to help select the most suitable human-AI interaction mode—automation, augmentation, or collaboration—based on task and user needs, supported by a scoring rubric and preliminary testing.

Contribution

It introduces a simple, criteria-driven approach and scoring rubric for selecting A2C modes, linking cognitive task analysis to practical decision-making in human-AI interaction.

Findings

Developed a set of key criteria from literature for A2C mode selection.

Created a scoring rubric to recommend A2C modes based on task characteristics.

Applied the criteria to real-world scientific tasks, gaining insights into decision points.

Abstract

Artificial intelligence (AI) tools are now prevalent in many knowledge work industries. As AI becomes more capable and interactive, there is a growing need for guidance on how to employ AI most effectively. The A2C framework (Tariq, Chhetri, Nepal & Paris, 2024) distinguishes three decision-making modes for engaging AI: automation (AI completes a task, including decision/action), augmentation (AI supports human to decide) and collaboration (iterative interaction between human and AI). However, selecting the appropriate mode for a specific application is not always straightforward. The goal of the present study was to compile and trial a simple set of criteria to support recommendations about appropriate A2C mode for a given application. Drawing on human factors and computer science literature, we identified key criteria related to elements of the task, impacts on worker and support needs. From these criteria we built a scoring rubric with recommendation for A2C mode. As a preliminary test of this approach, we applied the criteria to cognitive task analysis (CTA) outputs from three tasks in the science domain - genome annotation, biological collections curation and protein crystallization - which provided insights into worker decision points, challenges and expert strategies. This paper describes the method for connecting CTA to A2C, reflecting on the challenges and future directions.