Balancing Performance and Cost for Two-Hop Cooperative Communications: Stackelberg Game and Distributed Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a novel approach combining Stackelberg game theory and multi-agent reinforcement learning to optimize the trade-off between performance and cost in two-hop cooperative wireless networks with distributed decision-making.

Contribution

It formulates the problem as a Stackelberg game, proves equilibrium existence, and develops a MARL-based framework to approach equilibrium without channel state information.

Findings

The method achieves near-equilibrium performance within 2.9% of the optimal.

The approach outperforms alternative methods in time-invariant environments.

It effectively balances relay cooperation benefits and source cost-efficiency.

Abstract

This paper aims to balance performance and cost in a two-hop wireless cooperative communication network where the source and relays have contradictory optimization goals and make decisions in a distributed manner. This differs from most existing works that have typically assumed that source and relay nodes follow a schedule created implicitly by a central controller. We propose that the relays form an alliance in an attempt to maximize the benefit of relaying while the source aims to increase the channel capacity cost-effectively. To this end, we establish the trade problem as a Stackelberg game, and prove the existence of its equilibrium. Another important aspect is that we use multi-agent reinforcement learning (MARL) to approach the equilibrium in a situation where the instantaneous channel state information (CSI) is unavailable, and the source and relays do not have knowledge of each other's goal. A multi-agent deep deterministic policy gradient-based framework is designed, where the relay alliance and the source act as agents. Experiments demonstrate that the proposed method can obtain an acceptable performance that is close to the game-theoretic equilibrium for all players under time-invariant environments, which considerably outperforms its potential alternatives and is only about 2.9% away from the optimal solution.