Answering Count Queries with Explanatory Evidence

TL;DR

This paper introduces a novel approach for answering count queries in web search and question answering by inferring answers from multiple observations, providing explanations, and supporting semantic qualifiers, backed by a new dataset.

Contribution

It presents a new methodology that infers count answers from multiple observations, supports qualifiers, and offers explanatory evidence, advancing beyond prior single-answer or snippet-based methods.

Findings

Improved accuracy in count query answering.

Enhanced interpretability with explanatory evidence.

Demonstrated benefits across diverse query types.

Abstract

A challenging case in web search and question answering are count queries, such as \textit{"number of songs by John Lennon"}. Prior methods merely answer these with a single, and sometimes puzzling number or return a ranked list of text snippets with different numbers. This paper proposes a methodology for answering count queries with inference, contextualization and explanatory evidence. Unlike previous systems, our method infers final answers from multiple observations, supports semantic qualifiers for the counts, and provides evidence by enumerating representative instances. Experiments with a wide variety of queries show the benefits of our method. To promote further research on this underexplored topic, we release an annotated dataset of 5k queries with 200k relevant text spans.