Bregman Centroid Guided Cross-Entropy Method

TL;DR

This paper introduces Bregman Centroid Guided CEM, an enhancement to the Cross-Entropy Method that improves convergence and solution quality in multimodal optimization tasks within model-based reinforcement learning.

Contribution

It proposes a novel Bregman centroid approach for ensemble CEM, enabling better diversity and information aggregation with minimal computational overhead.

Findings

Improves convergence speed in synthetic benchmarks.

Enhances solution quality in navigation tasks.

Seamless integration into existing CEM pipelines.

Abstract

The Cross-Entropy Method (CEM) is a widely adopted trajectory optimizer in model-based reinforcement learning (MBRL), but its unimodal sampling strategy often leads to premature convergence in multimodal landscapes. In this work, we propose Bregman Centroid Guided CEM ($\mathcal{BC}$-EvoCEM), a lightweight enhancement to ensemble CEM that leverages $\textit{Bregman centroids}$ for principled information aggregation and diversity control. $\textbf{$\mathcal{BC}$-EvoCEM}$ computes a performance-weighted Bregman centroid across CEM workers and updates the least contributing ones by sampling within a trust region around the centroid. Leveraging the duality between Bregman divergences and exponential family distributions, we show that $\textbf{$\mathcal{BC}$-EvoCEM}$ integrates seamlessly into standard CEM pipelines with negligible overhead. Empirical results on synthetic benchmarks, a cluttered navigation task, and full MBRL pipelines demonstrate that $\textbf{$\mathcal{BC}$-EvoCEM}$ enhances both convergence and solution quality, providing a simple yet effective upgrade for CEM.