DDPG-E2E: A Novel Policy Gradient Approach for End-to-End Communication Systems

TL;DR

This paper introduces DDPG-E2E, a reinforcement learning-based end-to-end communication system that trains transceivers without prior channel knowledge, improving performance for large block lengths through a reward mechanism and CNN architecture.

Contribution

The paper proposes a novel DDPG-based E2E learning approach that eliminates the need for prior channel information and enhances performance with large block lengths.

Findings

Significant reduction in block error rate compared to existing methods

Effective training without prior channel knowledge

Improved performance with large block lengths

Abstract

The End-to-end (E2E) learning-based approach has great potential to reshape the existing communication systems by replacing the transceivers with deep neural networks. To this end, the E2E learning approach needs to assume the availability of prior channel information to mathematically formulate a differentiable channel layer for the backpropagation (BP) of the error gradients, thereby jointly optimizing the transmitter and the receiver. However, accurate and instantaneous channel state information is hardly obtained in practical wireless communication scenarios. Moreover, the existing E2E learning-based solutions exhibit limited performance in data transmissions with large block lengths. In this article, these practical issues are addressed by our proposed deep deterministic policy gradient-based E2E communication system. In particular, the proposed solution utilizes a reward feedback mechanism to train both the transmitter and the receiver, which alleviates the information loss of error gradients during BP. In addition, a convolutional neural network (CNN)-based architecture is developed to mitigate the curse of dimensionality problem when transmitting messages with large block lengths. Extensive simulations then demonstrate that our proposed solution can not only jointly train the transmitter and the receiver simultaneously without requiring the prior channel knowledge but also can obtain significant performance improvement on block error rate compared to state-of-the-art solutions.