SUper Team at SemEval-2016 Task 3: Building a feature-rich system for community question answering

TL;DR

This paper describes a feature-rich system for community question answering that achieved top results in SemEval-2016, combining semantic, lexical, metadata, and user features to improve answer selection.

Contribution

The paper introduces a comprehensive feature-based approach that significantly improves performance in community question answering tasks, especially highlighting the importance of metadata and semantic features.

Findings

Achieved best results on subtask C

Strong results on subtasks A and B

Metadata and semantic features are most impactful

Abstract

We present the system we built for participating in SemEval-2016 Task 3 on Community Question Answering. We achieved the best results on subtask C, and strong results on subtasks A and B, by combining a rich set of various types of features: semantic, lexical, metadata, and user-related. The most important group turned out to be the metadata for the question and for the comment, semantic vectors trained on QatarLiving data and similarities between the question and the comment for subtasks A and C, and between the original and the related question for Subtask B.