Constrained Optimal Fuel Consumption of HEVs under Observational Noise

TL;DR

This paper extends the constrained reinforcement learning framework for HEV fuel optimization by explicitly modeling observational noise in SOC and speed, demonstrating robustness and varying impacts on fuel efficiency.

Contribution

It introduces a robust CRL approach that accounts for sensor noise in SOC and speed, a novel aspect in HEV fuel consumption optimization.

Findings

Fuel consumption remains robust under noise variations.

SOC constraint satisfaction is maintained despite observational noise.

Noise impacts fuel efficiency differently depending on its source.

Abstract

In our prior work, we investigated the minimum fuel consumption of a hybrid electric vehicle (HEV) under a state-of-charge (SOC) balance constraint, assuming perfect SOC measurements and accurate reference speed profiles. The constrained optimal fuel consumption (COFC) problem was addressed using a constrained reinforcement learning (CRL) framework. However, in real-world scenarios, SOC readings are often corrupted by sensor noise, and reference speeds may deviate from actual driving conditions. To account for these imperfections, this study reformulates the COFC problem by explicitly incorporating observational noise in both SOC and reference speed. We adopt a robust CRL approach, where the noise is modeled as a uniform distribution, and employ a structured training procedure to ensure stability. The proposed method is evaluated through simulations on the Toyota Prius hybrid system (THS), using both the New European Driving Cycle (NEDC) and the Worldwide Harmonized Light Vehicles Test Cycle (WLTC). Results show that fuel consumption and SOC constraint satisfaction remain robust across varying noise levels. Furthermore, the analysis reveals that observational noise in SOC and speed can impact fuel consumption to different extents. To the best of our knowledge, this is the first study to explicitly examine how observational noise -- commonly encountered in dynamometer testing and predictive energy control (PEC) applications -- affects constrained optimal fuel consumption in HEVs.