DeepClean: Integrated Distortion Identification and Algorithm Selection for Rectifying Image Corruptions

TL;DR

DeepClean introduces a two-level sequential planning method for automated image distortion classification and rectification, improving object detection performance on COCO dataset by dynamically selecting suitable algorithms based on image embeddings.

Contribution

It presents a novel integrated approach combining distortion classification and algorithm selection in a single inference pass, adaptable to unseen algorithms and input images.

Findings

Outperforms three baseline methods on COCO object detection with distorted images.

Demonstrates dynamic reconfiguration capability based on input image content.

Shows generalizability to unseen candidate algorithms during inference.

Abstract

Distortion identification and rectification in images and videos is vital for achieving good performance in downstream vision applications. Instead of relying on fixed trial-and-error based image processing pipelines, we propose a two-level sequential planning approach for automated image distortion classification and rectification. At the higher level it detects the class of corruptions present in the input image, if any. The lower level selects a specific algorithm to be applied, from a set of externally provided candidate algorithms. The entire two-level setup runs in the form of a single forward pass during inference and it is to be queried iteratively until the retrieval of the original image. We demonstrate improvements compared to three baselines on the object detection task on COCO image dataset with rich set of distortions. The advantage of our approach is its dynamic reconfiguration, conditioned on the input image and generalisability to unseen candidate algorithms at inference time, since it relies only on the comparison of their output of the image embeddings.