CMU LiveMedQA at TREC 2017 LiveQA: A Consumer Health Question Answering System

TL;DR

This paper introduces LiveMedQA, a specialized consumer health question answering system with domain-specific features, evaluated in TREC 2017, highlighting its strengths and areas for improvement.

Contribution

The paper presents novel domain-specific features for health QA, including a question type analyzer, a knowledge graph, and a structure-aware searcher, enhancing answer quality.

Findings

Achieved an average score of 0.356 on a 3-point scale in TREC 2017.

Identified three major drawbacks in the current system.

Discussed potential solutions for system improvements.

Abstract

In this paper, we present LiveMedQA, a question answering system that is optimized for consumer health question. On top of the general QA system pipeline, we introduce several new features that aim to exploit domain-specific knowledge and entity structures for better performance. This includes a question type/focus analyzer based on deep text classification model, a tree-based knowledge graph for answer generation and a complementary structure-aware searcher for answer retrieval. LiveMedQA system is evaluated in the TREC 2017 LiveQA medical subtask, where it received an average score of 0.356 on a 3 point scale. Evaluation results revealed 3 substantial drawbacks in current LiveMedQA system, based on which we provide a detailed discussion and propose a few solutions that constitute the main focus of our subsequent work.