Not Quite 'Ask a Librarian': AI on the Nature, Value, and Future of LIS

TL;DR

This paper evaluates GPT-3's responses to complex LIS questions, revealing its strengths and limitations in generating insights about the field's nature, value, and future, and discusses AI's potential in research idea generation.

Contribution

It provides an LIS-specific assessment of GPT-3's capabilities and limitations, highlighting its potential and challenges in scholarly contexts.

Findings

GPT-3 offers diverse perspectives, from superficial to insightful.

Responses include both optimistic and worrisome visions of LIS's future.

AI language models can generate ideas but have limitations in depth and accuracy.

Abstract

AI language models trained on Web data generate prose that reflects human knowledge and public sentiments, but can also contain novel insights and predictions. We asked the world's best language model, GPT-3, fifteen difficult questions about the nature, value, and future of library and information science (LIS), topics that receive perennial attention from LIS scholars. We present highlights from its 45 different responses, which range from platitudes and caricatures to interesting perspectives and worrisome visions of the future, thus providing an LIS-tailored demonstration of the current performance of AI language models. We also reflect on the viability of using AI to forecast or generate research ideas in this way today. Finally, we have shared the full response log online for readers to consider and evaluate for themselves.