Addressing Community Question Answering in English and Arabic

TL;DR

This paper explores various feature-based machine learning models for question re-ranking in community QA, demonstrating the effectiveness of structural kernels and embeddings in both English and Arabic datasets, with notable improvements over baseline methods.

Contribution

It introduces the novel application of syntactic tree kernels to Arabic question re-ranking and compares multiple features and algorithms on multilingual community QA datasets.

Findings

Structural kernels are robust to noisy data.

Effective BoW and TK features improve re-ranking performance.

Achieved second-best results on SemEval-2016 tasks for English and Arabic.

Abstract

This paper studies the impact of different types of features applied to learning to re-rank questions in community Question Answering. We tested our models on two datasets released in SemEval-2016 Task 3 on "Community Question Answering". Task 3 targeted real-life Web fora both in English and Arabic. Our models include bag-of-words features (BoW), syntactic tree kernels (TKs), rank features, embeddings, and machine translation evaluation features. To the best of our knowledge, structural kernels have barely been applied to the question reranking task, where they have to model paraphrase relations. In the case of the English question re-ranking task, we compare our learning to rank (L2R) algorithms against a strong baseline given by the Google-generated ranking (GR). The results show that i) the shallow structures used in our TKs are robust enough to noisy data and ii) improving GR is possible, but effective BoW features and TKs along with an accurate model of GR features in the used L2R algorithm are required. In the case of the Arabic question re-ranking task, for the first time we applied tree kernels on syntactic trees of Arabic sentences. Our approaches to both tasks obtained the second best results on SemEval-2016 subtasks B on English and D on Arabic.