Sequence Aware SAC Control for Engine Fuel Consumption Optimization in Electrified Powertrain

TL;DR

This paper introduces a sequence-aware reinforcement learning framework using SAC, GRUs, and Decision Transformers to optimize engine control in hybrid electric trucks, achieving near-optimal fuel savings and robustness across diverse driving conditions.

Contribution

It develops a novel RL approach incorporating sequence modeling techniques for improved engine control in HEVs, outperforming traditional methods in fuel efficiency and adaptability.

Findings

SAC with DT-based actor and GRU critic achieved within 1.8% of DP in fuel savings.

Sequence-aware agents outperformed feedforward networks on unseen drive cycles.

Models demonstrated robustness and generalization across diverse initial states and drive conditions.

Abstract

As hybrid electric vehicles (HEVs) gain traction in heavy-duty trucks, adaptive and efficient energy management is critical for reducing fuel consumption while maintaining battery charge for long operation times. We present a new reinforcement learning (RL) framework based on the Soft Actor-Critic (SAC) algorithm to optimize engine control in series HEVs. We reformulate the control task as a sequential decision-making problem and enhance SAC by incorporating Gated Recurrent Units (GRUs) and Decision Transformers (DTs) into both actor and critic networks to capture temporal dependencies and improve planning over time. To evaluate robustness and generalization, we train the models under diverse initial battery states, drive cycle durations, power demands, and input sequence lengths. Experiments show that the SAC agent with a DT-based actor and GRU-based critic was within 1.8% of Dynamic Programming (DP) in fuel savings on the Highway Fuel Economy Test (HFET) cycle, while the SAC agent with GRUs in both actor and critic networks, and FFN actor-critic agent were within 3.16% and 3.43%, respectively. On unseen drive cycles (US06 and Heavy Heavy-Duty Diesel Truck (HHDDT) cruise segment), generalized sequence-aware agents consistently outperformed feedforward network (FFN)-based agents, highlighting their adaptability and robustness in real-world settings.