Exploring the Advantages of Dense-Vector to One-Hot Encoding of Intent Classes in Out-of-Scope Detection Tasks

TL;DR

This paper investigates how dense-vector encodings of intent classes can significantly improve out-of-scope detection in intent classification tasks, surpassing traditional one-hot encoding methods, and introduces a new algorithm for optimizing these encodings.

Contribution

It demonstrates that dense-vector encodings, even random ones, can outperform one-hot encodings in OOS detection and proposes a novel search algorithm for effective dense-vector encoding selection.

Findings

Dense-vector encodings create richer OOS space topologies.

Random dense-vector encodings outperform one-hot encodings by over 20%.

The proposed search algorithm shows promising initial results.

Abstract

This work explores the intrinsic limitations of the popular one-hot encoding method in classification of intents when detection of out-of-scope (OOS) inputs is required. Although recent work has shown that there can be significant improvements in OOS detection when the intent classes are represented as dense-vectors based on domain specific knowledge, we argue in this paper that such gains are more likely due to advantages of dense-vector to one-hot encoding methods in representing the complexity of the OOS space. We start by showing how dense-vector encodings can create OOS spaces with much richer topologies than one-hot encoding methods. We then demonstrate empirically, using four standard intent classification datasets, that knowledge-free, randomly generated dense-vector encodings of intent classes can yield massive, over 20% gains over one-hot encodings, and also outperform the previous, domain knowledge-based, SOTA of one of the datasets. We finish by describing a novel algorithm to search for good dense-vector encodings and present initial but promising experimental results of its use.