FM-EAC: Feature Model-based Enhanced Actor-Critic for Multi-Task Control in Dynamic Environments

TL;DR

FM-EAC is a novel reinforcement learning algorithm that combines model-based and model-free approaches with feature-based models to improve multi-task control and transferability in dynamic environments, demonstrated through urban and agricultural simulations.

Contribution

The paper introduces FM-EAC, a generalized algorithm integrating planning, acting, and learning with feature models and an enhanced actor-critic for better multi-task transferability.

Findings

FM-EAC outperforms state-of-the-art methods in simulations.

It offers customizable sub-networks for user-specific needs.

Demonstrates effectiveness in urban and agricultural scenarios.

Abstract

Model-based reinforcement learning (MBRL) and model-free reinforcement learning (MFRL) evolve along distinct paths but converge in the design of Dyna-Q [1]. However, modern RL methods still struggle with effective transferability across tasks and scenarios. Motivated by this limitation, we propose a generalized algorithm, Feature Model-Based Enhanced Actor-Critic (FM-EAC), that integrates planning, acting, and learning for multi-task control in dynamic environments. FM-EAC combines the strengths of MBRL and MFRL and improves generalizability through the use of novel feature-based models and an enhanced actor-critic framework. Simulations in both urban and agricultural applications demonstrate that FM-EAC consistently outperforms many state-of-the-art MBRL and MFRL methods. More importantly, different sub-networks can be customized within FM-EAC according to user-specific requirements.