Retrieving Comparative Arguments using Ensemble Methods and Neural Information Retrieval

TL;DR

This paper introduces ensemble and neural network methods, including BERT, for improved argument retrieval in response to comparative questions, achieving top performance in relevance and quality metrics.

Contribution

It combines decision tree ensembles with neural language models like BERT for ranking comparative arguments, advancing retrieval techniques for complex queries.

Findings

Outperformed baseline models in relevance and quality

Achieved first place in relevance and second in quality metrics

Demonstrated effectiveness of combining ensemble and neural models

Abstract

In this paper, we present a submission to the Touche lab's Task 2 on Argument Retrieval for Comparative Questions. Our team Katana supplies several approaches based on decision tree ensembles algorithms to rank comparative documents in accordance with their relevance and argumentative support. We use PyTerrier library to apply ensembles models to a ranking problem, considering statistical text features and features based on comparative structures. We also employ large contextualized language modelling techniques, such as BERT, to solve the proposed ranking task. To merge this technique with ranking modelling, we leverage neural ranking library OpenNIR. Our systems substantially outperforming the proposed baseline and scored first in relevance and second in quality according to the official metrics of the competition (for measure NDCG@5 score). Presented models could help to improve the performance of processing comparative queries in information retrieval and dialogue systems.