Few-shot Algorithm Assurance
Dang Nguyen, Sunil Gupta

TL;DR
This paper introduces a novel method for assessing deep learning model reliability under image distortions, using a Level Set Estimation classifier and synthetic data generation, especially effective in few-sample scenarios.
Contribution
It proposes a new classifier based on Level Set Estimation for model assurance under distortions and extends it to few-sample settings with a novel VAE for synthetic data generation.
Findings
Outperforms strong baselines on five benchmark datasets
Effective in few-sample scenarios with synthetic image generation
Accurately predicts model accuracy above thresholds under distortions
Abstract
In image classification tasks, deep learning models are vulnerable to image distortion. For successful deployment, it is important to identify distortion levels under which the model is usable i.e. its accuracy stays above a stipulated threshold. We refer to this problem as Model Assurance under Image Distortion, and formulate it as a classification task. Given a distortion level, our goal is to predict if the model's accuracy on the set of distorted images is greater than a threshold. We propose a novel classifier based on a Level Set Estimation (LSE) algorithm, which uses the LSE's mean and variance functions to form the classification rule. We further extend our method to a "few sample" setting where we can only acquire few real images to perform the model assurance process. Our idea is to generate extra synthetic images using a novel Conditional Variational Autoencoder model with…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMedical Imaging Techniques and Applications · Adversarial Robustness in Machine Learning · Advanced Radiotherapy Techniques
MethodsSparse Evolutionary Training
