Multiple View Generation and Classification of Mid-wave Infrared Images using Deep Learning

TL;DR

This paper introduces a deep learning approach for generating and classifying unseen viewpoints of mid-wave infrared images by understanding their semantic content and operating in a Riemannian feature space, improving over traditional blurry synthetic methods.

Contribution

It presents a novel method that models the manifold of infrared images in a non-linear Riemannian space, enabling better viewpoint generation and classification.

Findings

Generated views have good 3D semantic representations

Network operates in Riemannian rather than Euclidean space

Effective in low-shot infrared image classification

Abstract

We propose a novel study of generating unseen arbitrary viewpoints for infrared imagery in the non-linear feature subspace . Current methods use synthetic images and often result in blurry and distorted outputs. Our approach on the contrary understands the semantic information in natural images and encapsulates it such that our predicted unseen views possess good 3D representations. We further explore the non-linear feature subspace and conclude that our network does not operate in the Euclidean subspace but rather in the Riemannian subspace. It does not learn the geometric transformation for predicting the position of the pixel in the new image but rather learns the manifold. To this end, we use t-SNE visualisations to conduct a detailed analysis of our network and perform classification of generated images as a low-shot learning task.