Discriminative, Generative and Self-Supervised Approaches for Target-Agnostic Learning

TL;DR

This paper explores target-agnostic learning, a flexible approach that handles varying attribute sets for prediction, by surveying techniques, adapting algorithms, and evaluating their performance on diverse datasets.

Contribution

It introduces the task of target-agnostic learning, adapts existing methods for this setting, and provides extensive experimental evaluation on multiple data types.

Findings

Generative and self-supervised models perform well on target-agnostic tasks.

Algorithms show different strengths depending on data type.

Pseudo-likelihood theory relates these models for joint distribution inference.

Abstract

Supervised learning, characterized by both discriminative and generative learning, seeks to predict the values of single (or sometimes multiple) predefined target attributes based on a predefined set of predictor attributes. For applications where the information available and predictions to be made may vary from instance to instance, we propose the task of target-agnostic learning where arbitrary disjoint sets of attributes can be used for each of predictors and targets for each to-be-predicted instance. For this task, we survey a wide range of techniques available for handling missing values, self-supervised training and pseudo-likelihood training, and adapt them to a suite of algorithms that are suitable for the task. We conduct extensive experiments on this suite of algorithms on a large collection of categorical, continuous and discretized datasets, and report their performance in terms of both classification and regression errors. We also report the training and prediction time of these algorithms when handling large-scale datasets. Both generative and self-supervised learning models are shown to perform well at the task, although their characteristics towards the different types of data are quite different. Nevertheless, our derived theorem for the pseudo-likelihood theory also shows that they are related for inferring a joint distribution model based on the pseudo-likelihood training.