AutoRC: Improving BERT Based Relation Classification Models via Architecture Search

TL;DR

This paper introduces AutoRC, a neural architecture search approach to optimize BERT-based relation classification models by automatically selecting entity span identification, pooling operations, and feature interactions, leading to improved performance.

Contribution

The paper presents a comprehensive search space and employs NAS to automatically discover better BERT-based RC architectures, outperforming baseline models.

Findings

AutoRC finds architectures that outperform baseline BERT RC models.

The search space design is crucial for model performance.

AutoRC is efficient and effective across seven benchmark tasks.

Abstract

Although BERT based relation classification (RC) models have achieved significant improvements over the traditional deep learning models, it seems that no consensus can be reached on what is the optimal architecture. Firstly, there are multiple alternatives for entity span identification. Second, there are a collection of pooling operations to aggregate the representations of entities and contexts into fixed length vectors. Third, it is difficult to manually decide which feature vectors, including their interactions, are beneficial for classifying the relation types. In this work, we design a comprehensive search space for BERT based RC models and employ neural architecture search (NAS) method to automatically discover the design choices mentioned above. Experiments on seven benchmark RC tasks show that our method is efficient and effective in finding better architectures than the baseline BERT based RC model. Ablation study demonstrates the necessity of our search space design and the effectiveness of our search method.