Multiple Instance Verification

TL;DR

This paper introduces cross-attention pooling (CAP), a novel approach for multiple instance verification that significantly improves accuracy and key instance detection over existing MIL and Siamese methods.

Contribution

The paper proposes a new pooling method called cross-attention pooling (CAP) and two novel attention functions to enhance multiple instance verification tasks.

Findings

CAP outperforms SOTA MIL and baseline methods in accuracy

CAP improves key instance detection capabilities

New attention functions effectively distinguish similar instances

Abstract

We explore multiple instance verification, a problem setting in which a query instance is verified against a bag of target instances with heterogeneous, unknown relevancy. We show that naive adaptations of attention-based multiple instance learning (MIL) methods and standard verification methods like Siamese neural networks are unsuitable for this setting: directly combining state-of-the-art (SOTA) MIL methods and Siamese networks is shown to be no better, and sometimes significantly worse, than a simple baseline model. Postulating that this may be caused by the failure of the representation of the target bag to incorporate the query instance, we introduce a new pooling approach named "cross-attention pooling" (CAP). Under the CAP framework, we propose two novel attention functions to address the challenge of distinguishing between highly similar instances in a target bag. Through empirical studies on three different verification tasks, we demonstrate that CAP outperforms adaptations of SOTA MIL methods and the baseline by substantial margins, in terms of both classification accuracy and the ability to detect key instances. The superior ability to identify key instances is attributed to the new attention functions by ablation studies. We share our code at https://github.com/xxweka/MIV.