Nonstationary Discounted Stochastic Games under Prospect Theory with Applications to the Smart Grid

TL;DR

This paper studies nonstationary discounted stochastic games under prospect theory, establishing the existence of Nash equilibrium using a new technique, and applies it to smart grid energy management with simulation results.

Contribution

It introduces a novel approach to prove equilibrium existence in nonstationary prospect-theoretic stochastic games, extending to finite horizon and providing an algorithm for approximate solutions.

Findings

Existence of Markov Nash equilibrium under nonstationary conditions

Development of an algorithm for Markov ε-equilibrium

Application to smart grid energy management with simulation evidence

Abstract

This paper considers the discounted criterion of nonzero-sum decentralized stochastic games with prospect players. The state and action spaces are finite. The state transition probability is nonstationary. Each player independently controls their own Markov chain. The subjective behavior of players is described by the prospect theory (PT). Compared to the average criterion of stochastic games under PT studied firstly in 2018, we are concerned with the time value of utility, i.e., the utility should be discounted in the future. Since PT distorts the probability, the optimality equation that plays a significant role in proving the existence of equilibrium does not exist. On the other hand, the games change into Markov decision processes (MDPs) with nonstationary payoff function when fixing others' stationary Markov strategies, then the occupation measure and the linear programming of stationary MDPs are no longer suitable. Therefore, we explore a new technique by constructing the marginal distribution on the state-action pairs at any time, and establish the existence of Nash equilibrium. When the initial state is given, there exists a Markov Nash equilibrium. Furthermore, this novel technique can be extended to the finite horizon criterion. Then, we present an algorithm to find a Markov $\varepsilon$-equilibrium. Finally, the model is applied to a noncooperative stochastic game among prosumers who can produce and consume energy in the smart grid, and we give some simulation results.