Generative AI's aggregated knowledge versus web-based curated knowledge

TL;DR

This paper compares the effectiveness of generative AI and web-based search in knowledge retrieval, highlighting their respective strengths for different types of questions and user goals.

Contribution

It introduces a taxonomy distinguishing when generative AI or curated web search better serves user information needs based on experimental insights.

Findings

GenAI speeds up exploration and decision-making.

Web search excels at specific, less-known facts.

Different knowledge paradigms serve different user goals.

Abstract

his paper explores what kinds of questions are best served by the way generative AI (GenAI) using Large Language Models(LLMs) that aggregate and package knowledge, and when traditional curated web-sourced search results serve users better. An experiment compared product searches using ChatGPT, Google search engine, or both helped us understand more about the compelling nature of generated responses. The experiment showed GenAI can speed up some explorations and decisions. We describe how search can deepen the testing of facts, logic, and context. We show where existing and emerging knowledge paradigms can help knowledge exploration in different ways. Experimenting with searches, our probes showed the value for curated web search provides for very specific, less popularly-known knowledge. GenAI excelled at bringing together knowledge for broad, relatively well-known topics. The value of curated and aggregated knowledge for different kinds of knowledge reflected in different user goals. We developed a taxonomy to distinguishing when users are best served by these two approaches.