Learning from Complementary Features

TL;DR

This paper introduces Complementary Feature Learning (CFL), a new framework for predictive modeling using features that indicate what data is not, and proposes an information-theoretic estimation method with proven guarantees.

Contribution

The paper formulates CFL, a novel learning scenario using complementary features, and develops a graph-based estimation method with theoretical guarantees.

Findings

Effectively estimates OF values from CFs in real-world data

Improves prediction accuracy using complementary features

Provides a theoretically grounded estimation approach

Abstract

While precise data observation is essential for the learning processes of predictive models, it can be challenging owing to factors such as insufficient observation accuracy, high collection costs, and privacy constraints. In this paper, we examines cases where some qualitative features are unavailable as precise information indicating "what it is," but rather as complementary information indicating "what it is not." We refer to features defined by precise information as ordinary features (OFs) and those defined by complementary information as complementary features (CFs). We then formulate a new learning scenario termed Complementary Feature Learning (CFL), where predictive models are constructed using instances consisting of OFs and CFs. The simplest formalization of CFL applies conventional supervised learning directly using the observed values of CFs. However, this approach does not resolve the ambiguity associated with CFs, making learning challenging and complicating the interpretation of the predictive model's specific predictions. Therefore, we derive an objective function from an information-theoretic perspective to estimate the OF values corresponding to CFs and to predict output labels based on these estimations. Based on this objective function, we propose a theoretically guaranteed graph-based estimation method along with its practical approximation, for estimating OF values corresponding to CFs. The results of numerical experiments conducted with real-world data demonstrate that our proposed method effectively estimates OF values corresponding to CFs and predicts output labels.