Efficient AUC Optimization for Information Ranking Applications

TL;DR

This paper introduces a non-linear method using additive regression trees to optimize AUC, especially multi-class AUC, improving ranking performance in information retrieval systems.

Contribution

It presents an efficient non-linear approach for AUC optimization with additive regression trees, emphasizing multi-class AUC for ranking applications.

Findings

Non-linear approach performs comparably on binary relevance datasets.

Non-linear approach outperforms linear methods on multi-relevance datasets.

Method enhances evaluation of retrieval systems using AUC metrics.

Abstract

Adequate evaluation of an information retrieval system to estimate future performance is a crucial task. Area under the ROC curve (AUC) is widely used to evaluate the generalization of a retrieval system. However, the objective function optimized in many retrieval systems is the error rate and not the AUC value. This paper provides an efficient and effective non-linear approach to optimize AUC using additive regression trees, with a special emphasis on the use of multi-class AUC (MAUC) because multiple relevance levels are widely used in many ranking applications. Compared to a conventional linear approach, the performance of the non-linear approach is comparable on binary-relevance benchmark datasets and is better on multi-relevance benchmark datasets.