Hybrid Energy-Aware Reward Shaping: A Unified Lightweight Physics-Guided Methodology for Policy Optimization

TL;DR

This paper introduces H-EARS, a physics-guided reward shaping method that improves reinforcement learning efficiency and stability by integrating energy-aware regularization with potential-based reward shaping, suitable for safety-critical applications.

Contribution

The paper presents a unified, lightweight, physics-guided reward shaping framework with theoretical guarantees and practical validation, enabling efficient policy optimization without full system models.

Findings

Enhanced convergence and stability in RL algorithms.

Improved energy efficiency in control tasks.

Validated applicability in safety-critical vehicle simulations.

Abstract

Deep reinforcement learning excels in continuous control but often requires extensive exploration, while physics-based models demand complete equations and suffer cubic complexity. This study proposes Hybrid Energy-Aware Reward Shaping (H-EARS), unifying potential-based reward shaping with energy-aware action regularization. H-EARS constrains action magnitude while balancing task-specific and energy-based potentials via functional decomposition, achieving linear complexity O(n) by capturing dominant energy components without full dynamics. We establish a theoretical foundation including: (1) functional independence for separate task/energy optimization; (2) energy-based convergence acceleration; (3) convergence guarantees under function approximation; and (4) approximate potential error bounds. Lyapunov stability connections are analyzed as heuristic guides. Experiments across baselines show improved convergence, stability, and energy efficiency. Vehicle simulations validate applicability in safety-critical domains under extreme conditions. Results confirm that integrating lightweight physics priors enhances model-free RL without complete system models, enabling transfer from lab research to industrial applications.