Efficient Meta Subspace Optimization
Yoni Choukroun, Michael Katz

TL;DR
This paper introduces a Meta Subspace Optimization framework that dynamically determines subspaces during large-scale optimization, utilizing a low-dimensional meta optimizer and reinforcement learning to outperform existing methods.
Contribution
The paper proposes a novel MSO framework with a dynamic subspace selection mechanism and an RL-based policy learning approach, improving optimization efficiency.
Findings
Reinforcement learning policies outperform existing subspace methods.
The meta optimizer effectively adapts subspaces at each iteration.
The approach remains invariant to problem dimension.
Abstract
Subspace optimization methods have the attractive property of reducing large-scale optimization problems to a sequence of low-dimensional subspace optimization problems. However, existing subspace optimization frameworks adopt a fixed update policy of the subspace and therefore appear to be sub-optimal. In this paper, we propose a new \emph{Meta Subspace Optimization} (MSO) framework for large-scale optimization problems, which allows to determine the subspace matrix at each optimization iteration. In order to remain invariant to the optimization problem's dimension, we design an \emph{efficient} meta optimizer based on very low-dimensional subspace optimization coefficients, inducing a rule-based method that can significantly improve performance. Finally, we design and analyze a reinforcement learning (RL) procedure based on the subspace optimization dynamics whose learnt policies…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMetaheuristic Optimization Algorithms Research · Advanced Multi-Objective Optimization Algorithms · Optimal Experimental Design Methods
