Aggregation as Unsupervised Learning and its Evaluation

TL;DR

This paper introduces a novel unsupervised aggregation method that leverages intrinsic properties of unlabeled data, demonstrating superior performance over existing approaches and comparable results to linear regression.

Contribution

The paper presents a new unsupervised aggregation approach based on data properties, along with an empirical evaluation framework comparing it to other methods.

Findings

Our approach outperforms other data-agnostic and unsupervised aggregation methods.

It is nearly as effective as linear regression in predicting latent ground truth.

The evaluation framework assesses both property preservation and ground truth prediction.

Abstract

Regression uses supervised machine learning to find a model that combines several independent variables to predict a dependent variable based on ground truth (labeled) data, i.e., tuples of independent and dependent variables (labels). Similarly, aggregation also combines several independent variables to a dependent variable. The dependent variable should preserve properties of the independent variables, e.g., the ranking or relative distance of the independent variable tuples, and/or represent a latent ground truth that is a function of these independent variables. However, ground truth data is not available for finding the aggregation model. Consequently, aggregation models are data agnostic or can only be derived with unsupervised machine learning approaches. We introduce a novel unsupervised aggregation approach based on intrinsic properties of unlabeled training data, such as the cumulative probability distributions of the single independent variables and their mutual dependencies. We present an empirical evaluation framework that allows assessing the proposed approach against other aggregation approaches from two perspectives: (i) how well the aggregation output represents properties of the input tuples, and (ii) how well can aggregated output predict a latent ground truth. To this end, we use data sets for assessing supervised regression approaches that contain explicit ground truth labels. However, the ground truth is not used for deriving the aggregation models, but it allows for the assessment from a perspective (ii). More specifically, we use regression data sets from the UCI machine learning repository and benchmark several data-agnostic and unsupervised approaches for aggregation against ours. The benchmark results indicate that our approach outperforms the other data-agnostic and unsupervised aggregation approaches. It is almost on par with linear regression.