Digital Twin-based Control Co-Design of Full Vehicle Active Suspensions via Deep Reinforcement Learning

TL;DR

This paper introduces a digital twin-based control co-design framework utilizing deep reinforcement learning for full vehicle active suspensions, enabling adaptive, personalized optimization under uncertain conditions to improve ride comfort and stability.

Contribution

It develops a novel DT-enabled CCD framework combining DRL and uncertainty-aware model updating, with a multi-generation design strategy for self-improving suspension systems.

Findings

Achieved 43% reduction in control effort for mild driving conditions.

Achieved 52% reduction in control effort for aggressive driving.

Demonstrated smoother vehicle trajectories and enhanced stability.

Abstract

Active suspension systems are critical for enhancing vehicle comfort, safety, and stability, yet their performance is often limited by fixed hardware designs and control strategies that cannot adapt to uncertain and dynamic operating conditions. Recent advances in digital twins (DTs) and deep reinforcement learning (DRL) offer new opportunities for real-time, data-driven optimization across a vehicle's lifecycle. However, integrating these technologies into a unified framework remains an open challenge. This work presents a DT-based control co-design (CCD) framework for full-vehicle active suspensions using multi-generation design concepts. By integrating automatic differentiation into DRL, we jointly optimize physical suspension components and control policies under varying driver behaviors and environmental uncertainties. DRL also addresses the challenge of partial observability, where only limited states can be sensed and fed back to the controller, by learning optimal control actions directly from available sensor information. The framework incorporates model updating with quantile learning to capture data uncertainty, enabling real-time decision-making and adaptive learning from digital-physical interactions. The approach demonstrates personalized optimization of suspension systems under two distinct driving settings (mild and aggressive). Results show that the optimized systems achieve smoother trajectories and reduce control efforts by approximately 43% and 52% for mild and aggressive, respectively, while maintaining ride comfort and stability. Contributions include: developing a DT-enabled CCD framework integrating DRL and uncertainty-aware model updating for full-vehicle active suspensions, introducing a multi-generation design strategy for self-improving systems, and demonstrating personalized optimization of active suspension systems for distinct driver types.