HeroNet: A Hybrid Retrieval-Generation Network for Conversational Bots

TL;DR

HeroNet is a novel hybrid retrieval-generation network for conversational bots that combines multi-task learning, adversarial training, and prior knowledge integration to enhance response quality and efficiency.

Contribution

It introduces a hybrid model that unifies retrieval and generation tasks using lightweight adapters, adversarial training, and knowledge integration, advancing conversational bot capabilities.

Findings

Improved retrieval and generation performance on open datasets.

Reduced model size with lightweight adapters.

Effective integration of retrieval results into generation.

Abstract

Using natural language, Conversational Bot offers unprecedented ways to many challenges in areas such as information searching, item recommendation, and question answering. Existing bots are usually developed through retrieval-based or generative-based approaches, yet both of them have their own advantages and disadvantages. To assemble this two approaches, we propose a hybrid retrieval-generation network (HeroNet) with the three-fold ideas: 1). To produce high-quality sentence representations, HeroNet performs multi-task learning on two subtasks: Similar Queries Discovery and Query-Response Matching. Specifically, the retrieval performance is improved while the model size is reduced by training two lightweight, task-specific adapter modules that share only one underlying T5-Encoder model. 2). By introducing adversarial training, HeroNet is able to solve both retrieval\&generation tasks simultaneously while maximizing performance of each other. 3). The retrieval results are used as prior knowledge to improve the generation performance while the generative result are scored by the discriminator and their scores are integrated into the generator's cross-entropy loss function. The experimental results on a open dataset demonstrate the effectiveness of the HeroNet and our code is available at https://github.com/TempHero/HeroNet.git