A novel multivariate performance optimization method based on sparse coding and hyper-predictor learning

TL;DR

This paper introduces a new multivariate performance optimization approach using sparse coding and hyper-predictor learning, which effectively minimizes complex loss functions for tuples of data points.

Contribution

It presents a joint optimization framework for learning dictionaries, sparse codes, and prediction parameters to optimize multivariate performance measures.

Findings

Outperforms state-of-the-art algorithms on benchmark datasets.

Effectively minimizes complex multivariate loss functions.

Demonstrates the advantage of sparse coding in multivariate optimization.

Abstract

In this paper, we investigate the problem of optimization multivariate performance measures, and propose a novel algorithm for it. Different from traditional machine learning methods which optimize simple loss functions to learn prediction function, the problem studied in this paper is how to learn effective hyper-predictor for a tuple of data points, so that a complex loss function corresponding to a multivariate performance measure can be minimized. We propose to present the tuple of data points to a tuple of sparse codes via a dictionary, and then apply a linear function to compare a sparse code against a give candidate class label. To learn the dictionary, sparse codes, and parameter of the linear function, we propose a joint optimization problem. In this problem, the both the reconstruction error and sparsity of sparse code, and the upper bound of the complex loss function are minimized. Moreover, the upper bound of the loss function is approximated by the sparse codes and the linear function parameter. To optimize this problem, we develop an iterative algorithm based on descent gradient methods to learn the sparse codes and hyper-predictor parameter alternately. Experiment results on some benchmark data sets show the advantage of the proposed methods over other state-of-the-art algorithms.