Alternative Shapes of Modulation Schemes Detailed Exposition and Simulation Methodology

TL;DR

This paper comprehensively studies various modulation constellation designs using geometric, probabilistic, optimization, and machine learning methods, evaluating their performance under different channel conditions and hardware constraints.

Contribution

It introduces a unified framework for constellation design analysis and demonstrates the effectiveness of machine learning in optimizing multiple performance metrics simultaneously.

Findings

SER-optimal designs may not be energy-optimal.

Small SER trade-offs can lead to significant energy savings.

Machine learning enables flexible joint optimization of reliability and energy efficiency.

Abstract

Modulation constellation design is a core challenge in digital communications, especially under stringent demands on spectral efficiency, robustness, and energy consumption. Classical schemes like PSK and QAM, while analytically tractable, often lose optimality under realistic channels and nonlinear hardware constraints. This paper provides a unified study of constellation design from geometric, probabilistic, optimization, and machine learning perspectives, focusing on symbol error rate (SER), fading robustness, peak-to-average power ratio (PAPR), and energy efficiency. We evaluate classical, lattice-based, asymmetric, probabilistically shaped, Golden Angle, heuristic-optimized, and machine learning assisted constellations under AWGN and Rayleigh fading via large-scale Monte Carlo simulations. Incorporating PAPR-aware and power amplifier models reveals that SER-optimal designs are not always energy-optimal; small SER trade-offs can yield substantial energy savings. Machine learning approaches offer flexible joint optimization of reliability, robustness, and energy efficiency by embedding channel and hardware constraints into the learning objective.