Transforming NLU with Babylon: A Case Study in Development of Real-time, Edge-Efficient, Multi-Intent Translation System for Automated Drive-Thru Ordering

TL;DR

Babylon is a transformer-based NLU system designed for real-time, edge-efficient multi-intent understanding in noisy environments like drive-thru ordering, outperforming traditional models in accuracy, latency, and memory use.

Contribution

This work introduces Babylon, a novel transformer architecture that handles multi-intent translation and noise robustness for edge NLU applications, with optimized low-latency and memory footprint.

Findings

Babylon achieves superior accuracy-latency trade-offs compared to Flan-T5 and BART.

The LSTM-based token pooling reduces input length and improves robustness.

Babylon effectively manages multi-intent scenarios in noisy, real-time environments.

Abstract

Real-time conversational AI agents face challenges in performing Natural Language Understanding (NLU) in dynamic, outdoor environments like automated drive-thru systems. These settings require NLU models to handle background noise, diverse accents, and multi-intent queries while operating under strict latency and memory constraints on edge devices. Additionally, robustness to errors from upstream Automatic Speech Recognition (ASR) is crucial, as ASR outputs in these environments are often noisy. We introduce Babylon, a transformer-based architecture that tackles NLU as an intent translation task, converting natural language inputs into sequences of regular language units ('transcodes') that encode both intents and slot information. This formulation allows Babylon to manage multi-intent scenarios in a single dialogue turn. Furthermore, Babylon incorporates an LSTM-based token pooling mechanism to preprocess phoneme sequences, reducing input length and optimizing for low-latency, low-memory edge deployment. This also helps mitigate inaccuracies in ASR outputs, enhancing system robustness. While this work focuses on drive-thru ordering, Babylon's design extends to similar noise-prone scenarios, for e.g. ticketing kiosks. Our experiments show that Babylon achieves significantly better accuracy-latency-memory footprint trade-offs over typically employed NMT models like Flan-T5 and BART, demonstrating its effectiveness for real-time NLU in edge deployment settings.