Soft Quality-Diversity Optimization
Saeed Hedayatian, Stefanos Nikolaidis
TL;DR
This paper introduces Soft QD, a new formulation for Quality-Diversity optimization that avoids discretization, enabling scalable solutions in high-dimensional spaces, and presents a differentiable algorithm called SQUAD that performs competitively on benchmarks.
Contribution
The paper proposes Soft QD as an alternative to traditional discretization-based QD methods and develops SQUAD, a differentiable algorithm that scales better to high-dimensional problems.
Findings
SQUAD is competitive with state-of-the-art methods on benchmarks.
Soft QD demonstrates desirable properties like monotonicity.
The approach improves scalability to high-dimensional solution spaces.
Abstract
Quality-Diversity (QD) algorithms constitute a branch of optimization that is concerned with discovering a diverse and high-quality set of solutions to an optimization problem. Current QD methods commonly maintain diversity by dividing the behavior space into discrete regions, ensuring that solutions are distributed across different parts of the space. The QD problem is then solved by searching for the best solution in each region. This approach to QD optimization poses challenges in large solution spaces, where storing many solutions is impractical, and in high-dimensional behavior spaces, where discretization becomes ineffective due to the curse of dimensionality. We present an alternative framing of the QD problem, called \emph{Soft QD}, that sidesteps the need for discretizations. We validate this formulation by demonstrating its desirable properties, such as monotonicity, and by…
Peer Reviews
Decision·ICLR 2026 Poster
- The proposed new objective bypasses the issues arising from behavior space discretizing, and enables it to be optimized by gradient-based optimizers easily. - The proposed method performs well on DQD tasks with high-dimensional behavior spaces. - The paper is well-organized and easy to follow. The code is available, contributing to reproducibility.
- The experiments are limited. The proposed method is not evaluated on high-dimensional reinforcement learning tasks, which are a more important type of DQD problem. Consequently, it is not compared with state-of-the-art methods that utilize policy gradients effectively. - There are also other QD methods that do not discretize the behavior space (e.g., novelty search-based methods). They are not compared in the experiments. - It is unclear whether the outstanding performance comes from the defin
- Novel Problem Formulation: Shifting the QD objective from a discrete, archive-based metric to a continuous, integral-based "illumination" field is a significant and original theoretical contribution. It elegantly sidesteps the arbitrary nature of defining grid resolutions or pre-computing tessellations. - Scalability to High-Dimensional Behavior Spaces: The empirical results strongly support the core claim that this method handles higher-dimensional behavior spaces better than archive-based c
- Hyperparameter Sensitivity ($\gamma$): The method's performance and its trade-off between quality and diversity are heavily dependent on the kernel bandwidth $\gamma^2$ (Section 5.3). While this provides controllability, it also introduces a critical hyperparameter that might be difficult to tune a priori for new domains compared to setting a grid resolution. - Bounded Space Reliance: Appendix C.3 highlights that the logit transformation for bounded spaces is "critical" for success. This sugg
The idea of "illumination" for addressing the high-dimensionality is interesting. A differentiable method of SQUAD is appreciated.
1. The core idea of continuous behavior space “illumination” overlaps with Kent et al. (2022)’s continuous QD Score. However, the paper only notes Kent et al.’s work as an evaluation tool, a more indepth discussion should be provided to support its novelty. 2. The paper claims SQUAD scales to high-dimensional spaces, but LP domain tests only go up to 16-dimensional behavior spaces. The impact of solution count on computational efficiency and optimization performance is unexamined. 3. For baseli
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComplexity and Algorithms in Graphs · Constraint Satisfaction and Optimization · Vehicle Routing Optimization Methods