Leveraging Large Language Models for Realizing Truly Intelligent User Interfaces

TL;DR

This paper explores integrating large language models into scholarly knowledge interfaces to improve semantic organization, demonstrating practical implementation, best practices, and expert evaluation.

Contribution

It introduces a method for non-intrusive LLM integration into existing scholarly knowledge interfaces, highlighting practical experiences and evaluation results.

Findings

Successful integration of LLMs into scholarly interfaces

Identified best practices for LLM-guided knowledge transformation

Positive expert feedback on interface usability

Abstract

The number of published scholarly articles is growing at a significant rate, making scholarly knowledge organization increasingly important. Various approaches have been proposed to organize scholarly information, including describing scholarly knowledge semantically leveraging knowledge graphs. Transforming unstructured knowledge, presented within articles, to structured and semantically represented knowledge generally requires human intelligence and labor since natural language processing methods alone typically do not render sufficient precision and recall for many applications. With the recent developments of Large Language Models (LLMs), it becomes increasingly possible to provide truly intelligent user interfaces guiding humans in the transformation process. We present an approach to integrate non-intrusive LLMs guidance into existing user interfaces. More specifically, we integrate LLM-supported user interface components into an existing scholarly knowledge infrastructure. Additionally, we provide our experiences with LLM integration, detailing best practices and obstacles. Finally, we evaluate the approach using a small-scale user evaluation with domain experts.