Peer-to-Peer Energy Trading in Dairy Farms using Multi-Agent Reinforcement Learning

TL;DR

This paper explores how multi-agent reinforcement learning enhances peer-to-peer energy trading in dairy farms, leading to cost savings, increased revenue, and peak demand reduction in rural renewable energy systems.

Contribution

It introduces a novel combination of MARL algorithms with P2P trading mechanisms for decentralized energy management in rural communities.

Findings

DQN reduces electricity costs by 14.2% in Ireland and 5.16% in Finland.

PPO reduces peak demand by 55.5% in Ireland.

DQN increases electricity revenue by up to 12.73%.

Abstract

The integration of renewable energy resources in rural areas, such as dairy farming communities, enables decentralized energy management through Peer-to-Peer (P2P) energy trading. This research highlights the role of P2P trading in efficient energy distribution and its synergy with advanced optimization techniques. While traditional rule-based methods perform well under stable conditions, they struggle in dynamic environments. To address this, Multi-Agent Reinforcement Learning (MARL), specifically Proximal Policy Optimization (PPO) and Deep Q-Networks (DQN), is combined with community/distributed P2P trading mechanisms. By incorporating auction-based market clearing, a price advisor agent, and load and battery management, the approach achieves significant improvements. Results show that, compared to baseline models, DQN reduces electricity costs by 14.2% in Ireland and 5.16% in Finland, while increasing electricity revenue by 7.24% and 12.73%, respectively. PPO achieves the lowest peak hour demand, reducing it by 55.5% in Ireland, while DQN reduces peak hour demand by 50.0% in Ireland and 27.02% in Finland. These improvements are attributed to both MARL algorithms and P2P energy trading, which together results in electricity cost and peak hour demand reduction, and increase electricity selling revenue. This study highlights the complementary strengths of DQN, PPO, and P2P trading in achieving efficient, adaptable, and sustainable energy management in rural communities.