Synthesizing Machine Learning Programs with PAC Guarantees via Statistical Sketching

TL;DR

This paper introduces algorithms for synthesizing machine learning programs with high-probability statistical guarantees, combining program synthesis with statistical learning theory to ensure reliable ML components.

Contribution

It presents novel sketching algorithms that incorporate statistical guarantees into program synthesis involving machine learning components.

Findings

Successfully synthesized list processing programs with DNNs for image inputs

Achieved probabilistic correctness guarantees in image classification tasks

Demonstrated applicability to precision medicine case studies

Abstract

We study the problem of synthesizing programs that include machine learning components such as deep neural networks (DNNs). We focus on statistical properties, which are properties expected to hold with high probability -- e.g., that an image classification model correctly identifies people in images with high probability. We propose novel algorithms for sketching and synthesizing such programs by leveraging ideas from statistical learning theory to provide statistical soundness guarantees. We evaluate our approach on synthesizing list processing programs that include DNN components used to process image inputs, as well as case studies on image classification and on precision medicine. Our results demonstrate that our approach can be used to synthesize programs with probabilistic guarantees.