Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents

TL;DR

This paper presents a contrastive training approach that significantly improves zero-shot classification of semi-structured documents by leveraging layout and style information, enabling better generalization to unseen categories.

Contribution

The authors introduce a matching-based contrastive training method tailored for zero-shot semi-structured document classification, addressing layout and style cues.

Findings

Significant boost in Macro F1 score with the proposed pretraining.

Effective in both supervised and unsupervised zero-shot settings.

Outperforms baseline methods in semi-structured document classification.

Abstract

We investigate semi-structured document classification in a zero-shot setting. Classification of semi-structured documents is more challenging than that of standard unstructured documents, as positional, layout, and style information play a vital role in interpreting such documents. The standard classification setting where categories are fixed during both training and testing falls short in dynamic environments where new document categories could potentially emerge. We focus exclusively on the zero-shot setting where inference is done on new unseen classes. To address this task, we propose a matching-based approach that relies on a pairwise contrastive objective for both pretraining and fine-tuning. Our results show a significant boost in Macro F$_1$ from the proposed pretraining step in both supervised and unsupervised zero-shot settings.