Regularising Non-linear Models Using Feature Side-information

TL;DR

This paper introduces a framework that leverages feature side-information to regularize non-linear models, enhancing prediction accuracy by incorporating feature similarities during learning.

Contribution

It proposes a novel method to integrate feature side-information into the training of general models, improving predictive performance over traditional approaches.

Findings

Significant performance improvements on benchmark datasets.

Effective incorporation of feature similarities improves model regularization.

Outperforms baseline models that ignore feature side-information.

Abstract

Very often features come with their own vectorial descriptions which provide detailed information about their properties. We refer to these vectorial descriptions as feature side-information. In the standard learning scenario, input is represented as a vector of features and the feature side-information is most often ignored or used only for feature selection prior to model fitting. We believe that feature side-information which carries information about features intrinsic property will help improve model prediction if used in a proper way during learning process. In this paper, we propose a framework that allows for the incorporation of the feature side-information during the learning of very general model families to improve the prediction performance. We control the structures of the learned models so that they reflect features similarities as these are defined on the basis of the side-information. We perform experiments on a number of benchmark datasets which show significant predictive performance gains, over a number of baselines, as a result of the exploitation of the side-information.