An Approach to Evaluate User Interfaces in a Scholarly Knowledge Communication Domain

TL;DR

This paper presents a tailored approach for evaluating user interfaces in scholarly knowledge graphs, adapting the Cognitive Walkthrough method to domain-specific needs, demonstrated on the Open Research Knowledge Graph.

Contribution

It introduces a domain-specific adaptation of the Cognitive Walkthrough for user interface evaluation in scholarly knowledge communication.

Findings

Effective evaluation of scholarly knowledge graph interfaces

Tailored questions improve domain-specific usability assessment

Demonstrated on Open Research Knowledge Graph

Abstract

The amount of research articles produced every day is overwhelming: scholarly knowledge is getting harder to communicate and easier to get lost. A possible solution is to represent the information in knowledge graphs: structures representing knowledge in networks of entities, their semantic types, and relationships between them. But this solution has its own drawback: given its very specific task, it requires new methods for designing and evaluating user interfaces. In this paper, we propose an approach for user interface evaluation in the knowledge communication domain. We base our methodology on the well-established Cognitive Walkthough approach but employ a different set of questions, tailoring the method towards domain-specific needs. We demonstrate our approach on a scholarly knowledge graph implementation called Open Research Knowledge Graph (ORKG).