Adaptive Scaling for Sparse Detection in Information Extraction

TL;DR

This paper introduces an adaptive scaling algorithm that improves neural network detection models in sparse positive class scenarios by directly optimizing F-measure through dynamic cost-sensitive learning, leading to more effective training.

Contribution

It presents a novel adaptive scaling method that addresses positive sparsity and optimizes F-measure without extra hyper-parameters in information extraction tasks.

Findings

Enhanced detection performance in sparse positive class scenarios.

More stable and effective training of neural network models.

Theoretical framework for instance importance measurement.

Abstract

This paper focuses on detection tasks in information extraction, where positive instances are sparsely distributed and models are usually evaluated using F-measure on positive classes. These characteristics often result in deficient performance of neural network based detection models. In this paper, we propose adaptive scaling, an algorithm which can handle the positive sparsity problem and directly optimize over F-measure via dynamic cost-sensitive learning. To this end, we borrow the idea of marginal utility from economics and propose a theoretical framework for instance importance measuring without introducing any additional hyper-parameters. Experiments show that our algorithm leads to a more effective and stable training of neural network based detection models.