Active Learning with Tabular Language Models

TL;DR

This paper explores active learning strategies for tabular language models in industrial settings, demonstrating that cell-level diversity-based acquisition functions can reduce labeling effort effectively.

Contribution

It introduces and evaluates active learning methods tailored for tabular language models, highlighting the importance of cell-level diversity in reducing annotation costs.

Findings

Cell-level acquisition functions with diversity reduce labeling effort

Enforced table diversity can negatively impact performance

Open questions remain on computational efficiency and human annotator perspectives

Abstract

Despite recent advancements in tabular language model research, real-world applications are still challenging. In industry, there is an abundance of tables found in spreadsheets, but acquisition of substantial amounts of labels is expensive, since only experts can annotate the often highly technical and domain-specific tables. Active learning could potentially reduce labeling costs, however, so far there are no works related to active learning in conjunction with tabular language models. In this paper we investigate different acquisition functions in a real-world industrial tabular language model use case for sub-cell named entity recognition. Our results show that cell-level acquisition functions with built-in diversity can significantly reduce the labeling effort, while enforced table diversity is detrimental. We further see open fundamental questions concerning computational efficiency and the perspective of human annotators.