Representation Learning for Context-Dependent Decision-Making

TL;DR

This paper introduces an online representation learning algorithm for context-dependent decision-making, demonstrating its effectiveness in adapting to changing environments and outperforming traditional methods like Q-learning in a cognitive task.

Contribution

The paper presents a novel online algorithm that learns and transfers context-dependent representations, improving adaptability in sequential decision-making tasks.

Findings

The proposed algorithm outperforms existing non-adaptive methods.

It effectively transfers learned representations across contexts.

Application to the Wisconsin Card Sorting Task shows improved flexibility.

Abstract

Humans are capable of adjusting to changing environments flexibly and quickly. Empirical evidence has revealed that representation learning plays a crucial role in endowing humans with such a capability. Inspired by this observation, we study representation learning in the sequential decision-making scenario with contextual changes. We propose an online algorithm that is able to learn and transfer context-dependent representations and show that it significantly outperforms the existing ones that do not learn representations adaptively. As a case study, we apply our algorithm to the Wisconsin Card Sorting Task, a well-established test for the mental flexibility of humans in sequential decision-making. By comparing our algorithm with the standard Q-learning and Deep-Q learning algorithms, we demonstrate the benefits of adaptive representation learning.