Task-Oriented Convex Bilevel Optimization with Latent Feasibility

TL;DR

This paper introduces a convex bilevel optimization framework for learning and vision tasks, utilizing a latent feasibility constraint to incorporate richer task information and improve solution efficiency.

Contribution

It proposes a novel task-oriented energy as a latent constraint within a convex bilevel model, enhancing flexibility and incorporating domain knowledge.

Findings

Convergence of the proposed numerical strategy is theoretically established.

The method demonstrates stability under computational errors.

Numerical experiments validate theoretical results and practical effectiveness.

Abstract

This paper firstly proposes a convex bilevel optimization paradigm to formulate and optimize popular learning and vision problems in real-world scenarios. Different from conventional approaches, which directly design their iteration schemes based on given problem formulation, we introduce a task-oriented energy as our latent constraint which integrates richer task information. By explicitly re-characterizing the feasibility, we establish an efficient and flexible algorithmic framework to tackle convex models with both shrunken solution space and powerful auxiliary (based on domain knowledge and data distribution of the task). In theory, we present the convergence analysis of our latent feasibility re-characterization based numerical strategy. We also analyze the stability of the theoretical convergence under computational error perturbation. Extensive numerical experiments are conducted to verify our theoretical findings and evaluate the practical performance of our method on different applications.