Query Expansion for Survey Question Retrieval in the Social Sciences

TL;DR

This paper explores query expansion techniques using thesaurus and co-occurrence methods to enhance retrieval of social science survey questions, demonstrating that automatic expansion can outperform manual reformulation and baseline methods.

Contribution

It introduces and evaluates automatic query expansion approaches specifically for social science survey question retrieval in digital libraries.

Findings

Co-occurrence-based expansion improves retrieval quality.

Automatic expansion outperforms manual reformulation.

Enhanced retrieval results compared to baseline methods.

Abstract

In recent years, the importance of research data and the need to archive and to share it in the scientific community have increased enormously. This introduces a whole new set of challenges for digital libraries. In the social sciences typical research data sets consist of surveys and questionnaires. In this paper we focus on the use case of social science survey question reuse and on mechanisms to support users in the query formulation for data sets. We describe and evaluate thesaurus- and co-occurrence-based approaches for query expansion to improve retrieval quality in digital libraries and research data archives. The challenge here is to translate the information need and the underlying sociological phenomena into proper queries. As we can show retrieval quality can be improved by adding related terms to the queries. In a direct comparison automatically expanded queries using extracted co-occurring terms can provide better results than queries manually reformulated by a domain expert and better results than a keyword-based BM25 baseline.