Image Translation via Fine-grained Knowledge Transfer

TL;DR

This paper introduces an interpretable, knowledge-based image translation framework that uses a knowledge library and a fast search method to improve scalability and interpretability across various image translation tasks.

Contribution

It proposes a novel knowledge retrieval and transfer framework with a plug-and-play library and a fast search algorithm, enhancing interpretability and scalability in image translation.

Findings

Effective across multiple image-translation tasks

Demonstrates interpretability through backtracking experiments

Shows improved scalability with the BHKM search method

Abstract

Prevailing image-translation frameworks mostly seek to process images via the end-to-end style, which has achieved convincing results. Nonetheless, these methods lack interpretability and are not scalable on different image-translation tasks (e.g., style transfer, HDR, etc.). In this paper, we propose an interpretable knowledge-based image-translation framework, which realizes the image-translation through knowledge retrieval and transfer. In details, the framework constructs a plug-and-play and model-agnostic general purpose knowledge library, remembering task-specific styles, tones, texture patterns, etc. Furthermore, we present a fast ANN searching approach, Bandpass Hierarchical K-Means (BHKM), to cope with the difficulty of searching in the enormous knowledge library. Extensive experiments well demonstrate the effectiveness and feasibility of our framework in different image-translation tasks. In particular, backtracking experiments verify the interpretability of our method. Our code soon will be available at https://github.com/AceSix/Knowledge_Transfer.