Card Sorting Simulator: Augmenting Design of Logical Information Architectures with Large Language Models
Eduard Kuric, Peter Demcak, Matus Krajcovic

TL;DR
This paper introduces a prototype AI-powered tool using Large Language Models to simulate card sorting, aiming to provide automated feedback and improve the design of information architectures, validated through a comprehensive comparison with real user data.
Contribution
It presents the Card Sorting Simulator, the first tool leveraging LLMs for automated card sorting simulation, validated against extensive real-world data.
Findings
Good agreement between simulated and real clusterings
Inconsistencies due to mental model differences
Complexity of cards affects simulation accuracy
Abstract
Card sorting is a common ideation technique that elicits information on users' mental organization of content and functionality by having them sort items into categories. For more robust card sorting research, digital card sorting tools could benefit from providing quick automated feedback. Our objective of this research is to advance toward an instrument that applies artificial intelligence (AI) to augment card sorting. For this purpose, we develop the Card Sorting Simulator, a prototype tool that leverages Large Language Models (LLMs) to generate informative categorizations of cards. To illuminate how aligned the simulation is with card sorting by actual participants, and to inform the instrument's design decisions, we conducted a generalizability-focused comparative study. We obtained 28 pre-existing card sorting studies from real practitioners, comprising 1,399 participants, along…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSemantic Web and Ontologies · Service-Oriented Architecture and Web Services · Advanced Database Systems and Queries
