Towards Distortion-Predictable Embedding of Neural Networks

TL;DR

This paper introduces a new loss function for CNN embeddings that enables predictable mappings between features of original and distorted images, improving understanding and quantification of distortions in computer vision.

Contribution

A novel contrastive-based loss function is proposed to produce CNN embeddings with predictable relationships between distorted and original images.

Findings

Embeddings exhibit linear relations under distortions.

The method quantifies distortion levels through embedding mappings.

Improved robustness to non-linear distortions in CNN models.

Abstract

Current research in Computer Vision has shown that Convolutional Neural Networks (CNN) give state-of-the-art performance in many classification tasks and Computer Vision problems. The embedding of CNN, which is the internal representation produced by the last layer, can indirectly learn topological and relational properties. Moreover, by using a suitable loss function, CNN models can learn invariance to a wide range of non-linear distortions such as rotation, viewpoint angle or lighting condition. In this work, new insights are discovered about CNN embeddings and a new loss function is proposed, derived from the contrastive loss, that creates models with more predicable mappings and also quantifies distortions. In typical distortion-dependent methods, there is no simple relation between the features corresponding to one image and the features of this image distorted. Therefore, these methods require to feed-forward inputs under every distortions in order to find the corresponding features representations. Our contribution makes a step towards embeddings where features of distorted inputs are related and can be derived from each others by the intensity of the distortion.