Do We Need Online NLU Tools?

TL;DR

This paper evaluates and compares various online and open-source intent recognition tools for conversational AI, providing criteria for selecting the best approach based on a new evaluation dataset.

Contribution

It introduces criteria for choosing intent recognition algorithms and provides a comparative analysis of public NLU services and open-source methods.

Findings

Public NLU services vary significantly in performance.

Open-source algorithms can be competitive with commercial services.

The paper offers practical guidance for selecting intent recognition tools.

Abstract

The intent recognition is an essential algorithm of any conversational AI application. It is responsible for the classification of an input message into meaningful classes. In many bot development platforms, we can configure the NLU pipeline. Several intent recognition services are currently available as an API, or we choose from many open-source alternatives. However, there is no comparison of intent recognition services and open-source algorithms. Many factors make the selection of the right approach to the intent recognition challenging in practice. In this paper, we suggest criteria to choose the best intent recognition algorithm for an application. We present a dataset for evaluation. Finally, we compare selected public NLU services with selected open-source algorithms for intent recognition.