Online Learning of Temporal Dependencies for Sustainable Foraging Problem

TL;DR

This paper investigates online learning methods, including Neuro-Evolution and Deep Recurrent Q-Networks, with LSTM to develop sustainable foraging strategies in dynamic multi-agent environments, highlighting challenges in social dilemma management.

Contribution

It explores the application of online learning and LSTM-based temporal dependency modeling for sustainable foraging in multi-agent systems, revealing limitations in social dilemma resolution.

Findings

LSTM aids single-agent sustainable strategies

LSTM does not improve multi-agent social dilemma management

Online learning methods can develop sustainable foraging behaviors

Abstract

The sustainable foraging problem is a dynamic environment testbed for exploring the forms of agent cognition in dealing with social dilemmas in a multi-agent setting. The agents need to resist the temptation of individual rewards through foraging and choose the collective long-term goal of sustainability. We investigate methods of online learning in Neuro-Evolution and Deep Recurrent Q-Networks to enable agents to attempt the problem one-shot as is often required by wicked social problems. We further explore if learning temporal dependencies with Long Short-Term Memory may be able to aid the agents in developing sustainable foraging strategies in the long term. It was found that the integration of Long Short-Term Memory assisted agents in developing sustainable strategies for a single agent, however failed to assist agents in managing the social dilemma that arises in the multi-agent scenario.