Intent Classification on Low-Resource Languages with Query Similarity Search

TL;DR

The paper introduces a novel approach to intent classification in low-resource languages by framing it as a query similarity search, enabling zero-shot classification without extensive labeled data.

Contribution

It proposes casting intent classification as a query similarity search problem, improving performance in low-resource and zero-shot scenarios.

Findings

Effective intent classification in low-resource languages

Zero-shot classification performance achieved

Query similarity search outperforms traditional methods

Abstract

Intent classification is an important component of a functional Information Retrieval ecosystem. Many current approaches to intent classification, typically framed as a classification problem, can be problematic as intents are often hard to define and thus data can be difficult and expensive to annotate. The problem is exacerbated when we need to extend the intent classification system to support multiple and in particular low-resource languages. To address this, we propose casting intent classification as a query similarity search problem - we use previous example queries to define an intent, and a query similarity method to classify an incoming query based on the labels of its most similar queries in latent space. With the proposed approach, we are able to achieve reasonable intent classification performance for queries in low-resource languages in a zero-shot setting.