Unified BERT for Few-shot Natural Language Understanding

TL;DR

UBERT is a unified BERT-based model designed for diverse NLU tasks, leveraging a biaffine network to unify training and improve semantic understanding across multiple tasks, especially in few-shot settings.

Contribution

The paper introduces UBERT, a novel unified model that encodes various NLU tasks into a common framework using biaffine scoring, enhancing multi-task learning and few-shot performance.

Findings

Won first place in the 2022 AIWIN competition.

Unified approach effectively handles multiple NLU tasks.

Improved performance in few-shot multi-task scenarios.

Abstract

Even as pre-trained language models share a semantic encoder, natural language understanding suffers from a diversity of output schemas. In this paper, we propose UBERT, a unified bidirectional language understanding model based on BERT framework, which can universally model the training objects of different NLU tasks through a biaffine network. Specifically, UBERT encodes prior knowledge from various aspects, uniformly constructing learning representations across multiple NLU tasks, which is conducive to enhancing the ability to capture common semantic understanding. By using the biaffine to model scores pair of the start and end position of the original text, various classification and extraction structures can be converted into a universal, span-decoding approach. Experiments show that UBERT wins the first price in the 2022 AIWIN - World Artificial Intelligence Innovation Competition, Chinese insurance few-shot multi-task track, and realizes the unification of extensive information extraction and linguistic reasoning tasks.