ARQMath Lab: An Incubator for Semantic Formula Search in zbMATH Open?

TL;DR

This paper discusses the development of the ARQMath Lab as an incubator for improving semantic formula search in zbMATH, exploring index structures, entity linking, and formula retrieval methods to better satisfy mathematical information needs.

Contribution

It introduces the ARQMath evaluation framework and investigates formula retrieval techniques, including MOI-based search, to enhance mathematical information retrieval.

Findings

Manual runs provided insights into user needs and answer types.

MOI search shows promising potential despite low competition scores.

Further research into MOI is motivated by perceived quality of results.

Abstract

The zbMATH database contains more than 4 million bibliographic entries. We aim to provide easy access to these entries. Therefore, we maintain different index structures, including a formula index. To optimize the findability of the entries in our database, we continuously investigate new approaches to satisfy the information needs of our users. We believe that the findings from the ARQMath evaluation will generate new insights into which index structures are most suitable to satisfy mathematical information needs. Search engines, recommender systems, plagiarism checking software, and many other added-value services acting on databases such as the arXiv and zbMATH need to combine natural and formula language. One initial approach to address this challenge is to enrich the mostly unstructured document data via Entity Linking. The ARQMath Task at CLEF 2020 aims to tackle the problem of linking newly posted questions from Math Stack Exchange (MSE) to existing ones that were already answered by the community. To deeply understand MSE information needs, answer-, and formula types, we performed manual runs for tasks 1 and 2. Furthermore, we explored several formula retrieval methods: For task 2, such as fuzzy string search, k-nearest neighbors, and our recently introduced approach to retrieve Mathematical Objects of Interest (MOI) with textual search queries. The task results show that neither our automated methods nor our manual runs archived good scores in the competition. However, the perceived quality of the hits returned by the MOI search particularly motivates us to conduct further research about MOI.