Deep Learning-Based Image Kernel for Inductive Transfer

TL;DR

This paper introduces a deep learning-based image kernel derived from a Siamese network for effective classification in low-data target classes, leveraging transfer learning without extensive fine-tuning.

Contribution

It presents a novel kernel construction from a Siamese network trained on non-target classes, enabling high accuracy with minimal target data and partial fine-tuning.

Findings

Kernel-based SVM achieves good accuracy without fine-tuning.

Partial fine-tuning outperforms state-of-the-art methods.

Insights into class separation and learning process on multiple datasets.

Abstract

We propose a method to classify images from target classes with a small number of training examples based on transfer learning from non-target classes. Without using any more information than class labels for samples from non-target classes, we train a Siamese net to estimate the probability of two images to belong to the same class. With some post-processing, output of the Siamese net can be used to form a gram matrix of a Mercer kernel. Coupled with a support vector machine (SVM), such a kernel gave reasonable classification accuracy on target classes without any fine-tuning. When the Siamese net was only partially fine-tuned using a small number of samples from the target classes, the resulting classifier outperformed the state-of-the-art and other alternatives. We share class separation capabilities and insights into the learning process of such a kernel on MNIST, Dogs vs. Cats, and CIFAR-10 datasets.