Fine-grained Image-to-Image Transformation towards Visual Recognition

TL;DR

This paper introduces a GAN-based method for fine-grained image-to-image transformation that preserves identity during large deformations, improving recognition tasks in few-shot learning scenarios.

Contribution

A novel model combining identity disentanglement, constrained nonalignment, and adaptive identity modulation for identity-preserving transformations under large geometric deformations.

Findings

Outperforms state-of-the-art models in identity preservation.

Enhances fine-grained recognition accuracy in few-shot learning.

Effective in transforming images with large pose and viewpoint changes.

Abstract

Existing image-to-image transformation approaches primarily focus on synthesizing visually pleasing data. Generating images with correct identity labels is challenging yet much less explored. It is even more challenging to deal with image transformation tasks with large deformation in poses, viewpoints, or scales while preserving the identity, such as face rotation and object viewpoint morphing. In this paper, we aim at transforming an image with a fine-grained category to synthesize new images that preserve the identity of the input image, which can thereby benefit the subsequent fine-grained image recognition and few-shot learning tasks. The generated images, transformed with large geometric deformation, do not necessarily need to be of high visual quality but are required to maintain as much identity information as possible. To this end, we adopt a model based on generative adversarial networks to disentangle the identity related and unrelated factors of an image. In order to preserve the fine-grained contextual details of the input image during the deformable transformation, a constrained nonalignment connection method is proposed to construct learnable highways between intermediate convolution blocks in the generator. Moreover, an adaptive identity modulation mechanism is proposed to transfer the identity information into the output image effectively. Extensive experiments on the CompCars and Multi-PIE datasets demonstrate that our model preserves the identity of the generated images much better than the state-of-the-art image-to-image transformation models, and as a result significantly boosts the visual recognition performance in fine-grained few-shot learning.