Uncertainty-Aware Knowledge Transformers for Peer-to-Peer Energy Trading with Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a new uncertainty-aware prediction model integrated with multi-agent reinforcement learning for peer-to-peer energy trading, significantly improving cost efficiency, revenue, and grid demand management.

Contribution

It proposes a heteroscedastic probabilistic transformer-based prediction model (KTU) that explicitly quantifies uncertainty, enhancing decision-making in P2P energy trading environments.

Findings

Up to 5.7% reduction in energy purchase costs without P2P trading.

Up to 44.7% increase in electricity sales revenue with P2P trading.

Peak grid demand reduced by over 38% with the proposed approach.

Abstract

This paper presents a novel framework for Peer-to-Peer (P2P) energy trading that integrates uncertainty-aware prediction with multi-agent reinforcement learning (MARL), addressing a critical gap in current literature. In contrast to previous works relying on deterministic forecasts, the proposed approach employs a heteroscedastic probabilistic transformer-based prediction model called Knowledge Transformer with Uncertainty (KTU) to explicitly quantify prediction uncertainty, which is essential for robust decision-making in the stochastic environment of P2P energy trading. The KTU model leverages domain-specific features and is trained with a custom loss function that ensures reliable probabilistic forecasts and confidence intervals for each prediction. Integrating these uncertainty-aware forecasts into the MARL framework enables agents to optimize trading strategies with a clear understanding of risk and variability. Experimental results show that the uncertainty-aware Deep Q-Network (DQN) reduces energy purchase costs by up to 5.7% without P2P trading and 3.2% with P2P trading, while increasing electricity sales revenue by 6.4% and 44.7%, respectively. Additionally, peak hour grid demand is reduced by 38.8% without P2P and 45.6% with P2P. These improvements are even more pronounced when P2P trading is enabled, highlighting the synergy between advanced forecasting and market mechanisms for resilient, economically efficient energy communities.