Game Theory Meets Statistical Mechanics in Deep Learning Design

TL;DR

This paper introduces a novel deep learning framework that integrates game theory and statistical mechanics principles, enhancing feature extraction and classification by modeling neurons as players in a sequential cooperative game.

Contribution

It presents a unique approach that conceptualizes neural networks as sequential games with neurons evaluated via Shapley values, improving efficiency and accuracy in facial attribute tasks.

Findings

Outperforms traditional neural networks in accuracy

Demonstrates improved efficiency in training and inference

Effective in facial age estimation and gender classification

Abstract

We present a novel deep graphical representation that seamlessly merges principles of game theory with laws of statistical mechanics. It performs feature extraction, dimensionality reduction, and pattern classification within a single learning framework. Our approach draws an analogy between neurons in a network and players in a game theory model. Furthermore, each neuron viewed as a classical particle (subject to statistical physics' laws) is mapped to a set of actions representing specific activation value, and neural network layers are conceptualized as games in a sequential cooperative game theory setting. The feed-forward process in deep learning is interpreted as a sequential game, where each game comprises a set of players. During training, neurons are iteratively evaluated and filtered based on their contributions to a payoff function, which is quantified using the Shapley value driven by an energy function. Each set of neurons that significantly contributes to the payoff function forms a strong coalition. These neurons are the only ones permitted to propagate the information forward to the next layers. We applied this methodology to the task of facial age estimation and gender classification. Experimental results demonstrate that our approach outperforms both multi-layer perceptron and convolutional neural network models in terms of efficiency and accuracy.