Interactions with Generative Information Retrieval Systems

TL;DR

This paper explores how generative IR systems enable richer, multi-modal, and interactive user engagement, moving beyond traditional search interfaces to include natural language, images, videos, and sensors for more flexible information seeking.

Contribution

It provides a comprehensive overview of interaction modalities, feedback mechanisms, and emerging interface frameworks for generative IR systems, highlighting novel multi-modal and proactive interaction approaches.

Findings

Users can express needs through natural language, images, videos, and gestures.

Explicit and implicit feedback improve retrieval relevance.

Proactive systems enable context-aware recommendations and multi-party conversations.

Abstract

At its core, information access and seeking is an interactive process. In existing search engines, interactions are limited to a few pre-defined actions, such as "requery", "click on a document", "scrolling up/down", "going to the next result page", "leaving the search engine", etc. A major benefit of moving towards generative IR systems is enabling users with a richer expression of information need and feedback and free-form interactions in natural language and beyond. In other words, the actions users take are no longer limited by the clickable links and buttons available on the search engine result page and users can express themselves freely through natural language. This can go even beyond natural language, through images, videos, gestures, and sensors using multi-modal generative IR systems. This chapter briefly discusses the role of interaction in generative IR systems. We will first discuss different ways users can express their information needs by interacting with generative IR systems. We then explain how users can provide explicit or implicit feedback to generative IR systems and how they can consume such feedback. Next, we will cover how users interactively can refine retrieval results. We will expand upon mixed-initiative interactions and discuss clarification and preference elicitation in more detail. We then discuss proactive generative IR systems, including context-aware recommendation, following up past conversations, contributing to multi-party conversations, and feedback requests. Providing explanation is another interaction type that we briefly discuss in this chapter. We will also briefly describe multi-modal interactions in generative information retrieval. Finally, we describe emerging frameworks and solutions for user interfaces with generative AI systems.