Minimally Supervised Feature Selection for Classification (Master's Thesis, University Politehnica of Bucharest)

TL;DR

This thesis introduces a fast, minimally supervised feature selection method that outperforms established techniques, especially with limited labeled data, by selecting independent yet collectively strong features across diverse datasets.

Contribution

It proposes a novel feature selection algorithm requiring minimal labeled data, demonstrating superior speed and performance compared to traditional methods, and validates transfer learning across diverse datasets.

Findings

Unsupervised approach outperforms existing methods significantly.

Supervised approach is effective with very limited training data.

Method is validated on multiple datasets including YouTube-Objects and MNIST.

Abstract

In the context of the highly increasing number of features that are available nowadays we design a robust and fast method for feature selection. The method tries to select the most representative features that are independent from each other, but are strong together. We propose an algorithm that requires very limited labeled data (as few as one labeled frame per class) and can accommodate as many unlabeled samples. We also present here the supervised approach from which we started. We compare our two formulations with established methods like AdaBoost, SVM, Lasso, Elastic Net and FoBa and show that our method is much faster and it has constant training time. Moreover, the unsupervised approach outperforms all the methods with which we compared and the difference might be quite prominent. The supervised approach is in most cases better than the other methods, especially when the number of training shots is very limited. All that the algorithm needs is to choose from a pool of positively correlated features. The methods are evaluated on the Youtube-Objects dataset of videos and on MNIST digits dataset, while at training time we also used features obtained on CIFAR10 dataset and others pre-trained on ImageNet dataset. Thereby, we also proved that transfer learning is useful, even though the datasets differ very much: from low-resolution centered images from 10 classes, to high-resolution images with objects from 1000 classes occurring in different regions of the images or to very difficult videos with very high intraclass variance. 7