Concept Transfer Learning for Adaptive Language Understanding

TL;DR

This paper introduces a hierarchical concept transfer learning method for adaptive language understanding, improving semantic representation and transfer across tasks like domain adaptation and value set mismatch.

Contribution

It proposes a novel hierarchical semantic representation of concepts and transfer learning approaches that enhance language understanding adaptation.

Findings

Achieved state-of-the-art F1-score of 96.08% on ATIS.

Validated efficiency and effectiveness through empirical studies.

Improved performance on LU benchmarks using only lexicon features.

Abstract

Concept definition is important in language understanding (LU) adaptation since literal definition difference can easily lead to data sparsity even if different data sets are actually semantically correlated. To address this issue, in this paper, a novel concept transfer learning approach is proposed. Here, substructures within literal concept definition are investigated to reveal the relationship between concepts. A hierarchical semantic representation for concepts is proposed, where a semantic slot is represented as a composition of {\em atomic concepts}. Based on this new hierarchical representation, transfer learning approaches are developed for adaptive LU. The approaches are applied to two tasks: value set mismatch and domain adaptation, and evaluated on two LU benchmarks: ATIS and DSTC 2\&3. Thorough empirical studies validate both the efficiency and effectiveness of the proposed method. In particular, we achieve state-of-the-art performance ($F_1$-score 96.08\%) on ATIS by only using lexicon features.