Classification Betters Regression in Query-based Multi-document Summarisation Techniques for Question Answering: Macquarie University at BioASQ7b

TL;DR

This paper explores classification-based methods, including reinforcement learning, for query-based multi-document summarisation in biomedical question answering, comparing their effectiveness to regression approaches and analyzing metric correlations.

Contribution

It introduces classification approaches with reinforcement learning for summarisation, advancing beyond previous regression-based methods in biomedical QA.

Findings

Classification approaches outperform regression in summarisation quality.

Reinforcement learning improves answer relevance and coherence.

ROUGE metrics show varying correlation with human scores.

Abstract

Task B Phase B of the 2019 BioASQ challenge focuses on biomedical question answering. Macquarie University's participation applies query-based multi-document extractive summarisation techniques to generate a multi-sentence answer given the question and the set of relevant snippets. In past participation we explored the use of regression approaches using deep learning architectures and a simple policy gradient architecture. For the 2019 challenge we experiment with the use of classification approaches with and without reinforcement learning. In addition, we conduct a correlation analysis between various ROUGE metrics and the BioASQ human evaluation scores.