An Efficient Task-Oriented Dialogue Policy: Evolutionary Reinforcement Learning Injected by Elite Individuals

TL;DR

This paper introduces a novel method combining evolutionary algorithms with deep reinforcement learning, enhanced by elite individual injection, to improve dialogue policy optimization efficiency and performance in task-oriented systems.

Contribution

It proposes an innovative integration of EA and DRL with an elite injection mechanism to accelerate convergence and enhance exploration in dialogue policy learning.

Findings

Significant performance improvements across four datasets.

Reduced exploration time due to elite individual injection.

Effective balance between exploration and exploitation achieved.

Abstract

Deep Reinforcement Learning (DRL) is widely used in task-oriented dialogue systems to optimize dialogue policy, but it struggles to balance exploration and exploitation due to the high dimensionality of state and action spaces. This challenge often results in local optima or poor convergence. Evolutionary Algorithms (EAs) have been proven to effectively explore the solution space of neural networks by maintaining population diversity. Inspired by this, we innovatively combine the global search capabilities of EA with the local optimization of DRL to achieve a balance between exploration and exploitation. Nevertheless, the inherent flexibility of natural language in dialogue tasks complicates this direct integration, leading to prolonged evolutionary times. Thus, we further propose an elite individual injection mechanism to enhance EA's search efficiency by adaptively introducing best-performing individuals into the population. Experiments across four datasets show that our approach significantly improves the balance between exploration and exploitation, boosting performance. Moreover, the effectiveness of the EII mechanism in reducing exploration time has been demonstrated, achieving an efficient integration of EA and DRL on task-oriented dialogue policy tasks.