Generator Assisted Mixture of Experts For Feature Acquisition in Batch

TL;DR

This paper introduces a batch feature acquisition method using a generator-assisted mixture of experts, enabling cost-effective feature selection with improved accuracy trade-offs, suitable for time-sensitive applications.

Contribution

It proposes a novel batch feature acquisition framework leveraging feature generation, data partitioning, and a mixture of experts model, advancing beyond sequential methods.

Findings

Outperforms existing methods in accuracy-cost trade-off.

Uses feature generator to reduce oracle query costs.

Employs data partitioning and mixture of experts for tractability.

Abstract

Given a set of observations, feature acquisition is about finding the subset of unobserved features which would enhance accuracy. Such problems have been explored in a sequential setting in prior work. Here, the model receives feedback from every new feature acquired and chooses to explore more features or to predict. However, sequential acquisition is not feasible in some settings where time is of the essence. We consider the problem of feature acquisition in batch, where the subset of features to be queried in batch is chosen based on the currently observed features, and then acquired as a batch, followed by prediction. We solve this problem using several technical innovations. First, we use a feature generator to draw a subset of the synthetic features for some examples, which reduces the cost of oracle queries. Second, to make the feature acquisition problem tractable for the large heterogeneous observed features, we partition the data into buckets, by borrowing tools from locality sensitive hashing and then train a mixture of experts model. Third, we design a tractable lower bound of the original objective. We use a greedy algorithm combined with model training to solve the underlying problem. Experiments with four datasets show that our approach outperforms these methods in terms of trade-off between accuracy and feature acquisition cost.