Optimistic Robust Optimization With Applications To Machine Learning

TL;DR

This paper introduces an optimistic approach to robust optimization, demonstrating its applications in linear programming and machine learning, offering less conservative solutions and new interpretations for regularization and noise handling.

Contribution

It presents a novel optimistic perspective on robust optimization, connecting it to machine learning regularization and outlier management, with practical solution methods.

Findings

Optimistic robust optimization reduces conservatism in linear programming.

It offers new interpretations for sparsity-inducing regularization schemes.

The approach effectively handles outliers and noise in machine learning models.

Abstract

Robust Optimization has traditionally taken a pessimistic, or worst-case viewpoint of uncertainty which is motivated by a desire to find sets of optimal policies that maintain feasibility under a variety of operating conditions. In this paper, we explore an optimistic, or best-case view of uncertainty and show that it can be a fruitful approach. We show that these techniques can be used to address a wide variety of problems. First, we apply our methods in the context of robust linear programming, providing a method for reducing conservatism in intuitive ways that encode economically realistic modeling assumptions. Second, we look at problems in machine learning and find that this approach is strongly connected to the existing literature. Specifically, we provide a new interpretation for popular sparsity inducing non-convex regularization schemes. Additionally, we show that successful approaches for dealing with outliers and noise can be interpreted as optimistic robust optimization problems. Although many of the problems resulting from our approach are non-convex, we find that DCA or DCA-like optimization approaches can be intuitive and efficient.