Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey

TL;DR

This survey reviews how Large Language Models address ambiguity in conversational question answering, categorizing disambiguation methods, analyzing datasets, and outlining future research directions for more reliable NLP systems.

Contribution

It provides a comprehensive categorization and analysis of disambiguation approaches in LLM-based conversational QA, including datasets and future challenges.

Findings

Categorized disambiguation techniques with pros and cons

Reviewed datasets for ambiguity detection and resolution

Identified open problems and future research directions

Abstract

Ambiguity remains a fundamental challenge in Natural Language Processing (NLP) due to the inherent complexity and flexibility of human language. With the advent of Large Language Models (LLMs), addressing ambiguity has become even more critical due to their expanded capabilities and applications. In the context of Conversational Question Answering (CQA), this paper explores the definition, forms, and implications of ambiguity for language driven systems, particularly in the context of LLMs. We define key terms and concepts, categorize various disambiguation approaches enabled by LLMs, and provide a comparative analysis of their advantages and disadvantages. We also explore publicly available datasets for benchmarking ambiguity detection and resolution techniques and highlight their relevance for ongoing research. Finally, we identify open problems and future research directions, especially in agentic settings, proposing areas for further investigation. By offering a comprehensive review of current research on ambiguities and disambiguation with LLMs, we aim to contribute to the development of more robust and reliable LLM-based systems.