Minimax Optimization: The Case of Convex-Submodular

TL;DR

This paper studies convex-submodular minimax problems involving both continuous and discrete variables, introduces new algorithms, and analyzes their convergence, complexity, and solution quality, with applications in machine learning.

Contribution

It introduces the class of convex-submodular minimax problems, develops algorithms for them, and provides theoretical analysis of their convergence and complexity.

Findings

Algorithms achieve provable convergence rates.

Methods effectively solve mixed continuous-discrete minimax problems.

Numerical experiments demonstrate practical effectiveness.

Abstract

Minimax optimization has been central in addressing various applications in machine learning, game theory, and control theory. Prior literature has thus far mainly focused on studying such problems in the continuous domain, e.g., convex-concave minimax optimization is now understood to a significant extent. Nevertheless, minimax problems extend far beyond the continuous domain to mixed continuous-discrete domains or even fully discrete domains. In this paper, we study mixed continuous-discrete minimax problems where the minimization is over a continuous variable belonging to Euclidean space and the maximization is over subsets of a given ground set. We introduce the class of convex-submodular minimax problems, where the objective is convex with respect to the continuous variable and submodular with respect to the discrete variable. Even though such problems appear frequently in machine learning applications, little is known about how to address them from algorithmic and theoretical perspectives. For such problems, we first show that obtaining saddle points are hard up to any approximation, and thus introduce new notions of (near-) optimality. We then provide several algorithmic procedures for solving convex and monotone-submodular minimax problems and characterize their convergence rates, computational complexity, and quality of the final solution according to our notions of optimally. Our proposed algorithms are iterative and combine tools from both discrete and continuous optimization. Finally, we provide numerical experiments to showcase the effectiveness of our purposed methods.