Semantic Answer Type Prediction using BERT: IAI at the ISWC SMART Task 2020

TL;DR

This paper evaluates BERT's effectiveness in answer type prediction within the ISWC SMART task, showing strong results for coarse types and significant improvements for fine-grained types over traditional methods.

Contribution

The study compares neural transformer models like BERT with traditional methods for answer type prediction, highlighting BERT's advantages in fine-grained classification.

Findings

Coarse-grained answer types achieved over 95% accuracy with standard methods.

BERT provides marginal gains for coarse types but significantly outperforms previous approaches for fine-grained types.

Transformer models excel in detailed answer type prediction tasks.

Abstract

This paper summarizes our participation in the SMART Task of the ISWC 2020 Challenge. A particular question we are interested in answering is how well neural methods, and specifically transformer models, such as BERT, perform on the answer type prediction task compared to traditional approaches. Our main finding is that coarse-grained answer types can be identified effectively with standard text classification methods, with over 95% accuracy, and BERT can bring only marginal improvements. For fine-grained type detection, on the other hand, BERT clearly outperforms previous retrieval-based approaches.