Human-AI Collaborative Taxonomy Construction: A Case Study in Profession-Specific Writing Assistants

TL;DR

This paper presents a human-AI collaborative approach to develop domain-specific writing taxonomies, enhancing LLM-based writing assistants for professional contexts through iterative expert feedback and interaction.

Contribution

It introduces a novel methodology combining expert feedback and iterative LLM interactions to create tailored taxonomies for domain-specific writing support.

Findings

Initial study highlights limitations of current LLMs in domain-specific writing

Proposed approach integrates expert feedback for taxonomy refinement

Aims to validate improved LLM-assisted writing in professional settings

Abstract

Large Language Models (LLMs) have assisted humans in several writing tasks, including text revision and story generation. However, their effectiveness in supporting domain-specific writing, particularly in business contexts, is relatively less explored. Our formative study with industry professionals revealed the limitations in current LLMs' understanding of the nuances in such domain-specific writing. To address this gap, we propose an approach of human-AI collaborative taxonomy development to perform as a guideline for domain-specific writing assistants. This method integrates iterative feedback from domain experts and multiple interactions between these experts and LLMs to refine the taxonomy. Through larger-scale experiments, we aim to validate this methodology and thus improve LLM-powered writing assistance, tailoring it to meet the unique requirements of different stakeholder needs.