Optimization Proxies using Limited Labeled Data and Training Time -- A Semi-Supervised Bayesian Neural Network Approach

TL;DR

This paper introduces a semi-supervised Bayesian Neural Network approach for optimization proxies that effectively handles scarce labeled data and limited training time, outperforming traditional DNNs in constrained energy system problems.

Contribution

The paper presents a novel semi-supervised Bayesian Neural Network framework that improves optimization proxy performance under data scarcity and training constraints, with probabilistic confidence bounds.

Findings

Up to tenfold reduction in maximum equality gap.

Halving of inequality gaps in optimization.

Effective probabilistic confidence bounds with limited validation data.

Abstract

Constrained optimization problems arise in various engineering systems such as inventory management and power grids. Standard deep neural network (DNN) based machine learning proxies are ineffective in practical settings where labeled data is scarce and training times are limited. We propose a semi-supervised Bayesian Neural Networks (BNNs) based optimization proxy for this complex regime, wherein training commences in a sandwiched fashion, alternating between a supervised learning step for minimizing cost, and an unsupervised learning step for enforcing constraint feasibility. We show that the proposed semi-supervised BNN outperforms DNN architectures on important non-convex constrained optimization problems from energy network operations, achieving up to a tenfold reduction in expected maximum equality gap and halving the inequality gaps. Further, the BNN's ability to provide posterior samples is leveraged to construct practically meaningful probabilistic confidence bounds on performance using a limited validation data, unlike prior methods. The implementation code for this study is available at: https://github.com/kaarthiksundar/BNN-OPF/.