# Bayesian reinforcement learning for adaptive control of energy recuperation in hydraulic excavator arms

## TL;DR

This paper introduces a new adaptive control system for hydraulic excavators that uses Bayesian reinforcement learning to improve energy efficiency while maintaining safety.

## Contribution

The novel integration of Bayesian inference and reinforcement learning enables adaptive energy recuperation in uncertain hydraulic excavator operations.

## Key findings

- The framework models system dynamics and accounts for uncertainties like soil resistance and sensor noise.
- A Bayesian particle filter estimates latent states, enabling belief-space reinforcement learning for control decisions.
- A safety-projection layer enforces strict operational constraints during real-time control adjustments.

## Abstract

Hydraulic excavators are among the most energy-intensive machines in construction and mining, with conventional hydraulic systems often operating under fixed pressure and flow settings that lead to significant energy loss. Improving energy efficiency while ensuring safety and adaptability under uncertain operating conditions remains a critical challenge. This study proposes a novel adaptive control framework that integrates Bayesian inference with reinforcement learning (RL) to enhance energy recuperation in hydraulic excavator arms. The framework explicitly models system dynamics, including hydraulic cylinders, pumps, valves, and accumulators, while accounting for uncertainties from soil resistance, temperature-dependent viscosity, component wear, and sensor noise. A Bayesian particle filter is employed to continuously estimate latent states such as soil resistance multipliers and accumulator pre-charge offsets, enabling belief-space reinforcement learning to make informed control decisions. The learned control policy adjusts pump pressure and valve commands in real time, while a safety-projection layer enforces strict operational constraints (5–35 MPa hydraulic pressure, 12–28 MPa accumulator window, valve rate limits, and section-level relief protections).