Cascade Attribute Learning Network

TL;DR

The paper introduces CALNet, a modular reinforcement learning framework that learns and combines task attributes separately, enabling zero-shot generalization to unseen tasks by assembling learned attribute modules.

Contribution

It presents a novel attribute learning approach in RL and a cascading network architecture for modular attribute assembly, improving transferability and generalization.

Findings

Successfully learned attributes in control tasks involving time, position, velocity, and acceleration.

Enabled zero-shot transfer to unseen tasks by assembling attribute modules.

Validated on diverse control problems demonstrating effective attribute learning and assembly.

Abstract

We propose the cascade attribute learning network (CALNet), which can learn attributes in a control task separately and assemble them together. Our contribution is twofold: first we propose attribute learning in reinforcement learning (RL). Attributes used to be modeled using constraint functions or terms in the objective function, making it hard to transfer. Attribute learning, on the other hand, models these task properties as modules in the policy network. We also propose using novel cascading compensative networks in the CALNet to learn and assemble attributes. Using the CALNet, one can zero shoot an unseen task by separately learning all its attributes, and assembling the attribute modules. We have validated the capacity of our model on a wide variety of control problems with attributes in time, position, velocity and acceleration phases.