Unsupervised Variational Translator for Bridging Image Restoration and High-Level Vision Tasks

TL;DR

This paper introduces VaT, an unsupervised variational translator that bridges image restoration and high-level vision tasks without retraining existing models, improving performance in degraded environments.

Contribution

The paper presents a novel unsupervised method that uses variational inference to connect restoration and high-level vision networks without retraining, enabling effective image enhancement for downstream tasks.

Findings

Outperforms state-of-the-art unsupervised methods in dehazing and low-light enhancement.

Achieves comparable or better results than supervised methods in complex real-world scenarios.

Maintains content fidelity while enhancing high-level vision task performance.

Abstract

Recent research tries to extend image restoration capabilities from human perception to machine perception, thereby enhancing the performance of high-level vision tasks in degraded environments. These methods, primarily based on supervised learning, typically involve the retraining of restoration networks or high-level vision networks. However, collecting paired data in real-world scenarios and retraining large-scale models are challenge. To this end, we propose an unsupervised learning method called \textbf{Va}riational \textbf{T}ranslator (VaT), which does not require retraining existing restoration and high-level vision networks. Instead, it establishes a lightweight network that serves as an intermediate bridge between them. By variational inference, VaT approximates the joint distribution of restoration output and high-level vision input, dividing the optimization objective into preserving content and maximizing marginal likelihood associated with high-level vision tasks. By cleverly leveraging self-training paradigms, VaT achieves the above optimization objective without requiring labels. As a result, the translated images maintain a close resemblance to their original content while also demonstrating exceptional performance on high-level vision tasks. Extensive experiments in dehazing and low-light enhancement for detection and classification show the superiority of our method over other state-of-the-art unsupervised counterparts, even significantly surpassing supervised methods in some complex real-world scenarios.Code is available at https://github.com/Fire-friend/VaT.