Improving Document Image Understanding with Reinforcement Finetuning

TL;DR

This paper introduces a reinforcement learning-based finetuning method to enhance document image understanding, particularly effective with limited training data, by treating the information extraction model as a policy network and optimizing it with reward functions.

Contribution

The paper presents a novel reinforcement learning finetuning approach for document image understanding, improving performance in low-data scenarios.

Findings

Consistent performance improvements on four datasets.

Enhanced extraction accuracy with limited labeled data.

Effective use of expert feedback for model improvement.

Abstract

Successful Artificial Intelligence systems often require numerous labeled data to extract information from document images. In this paper, we investigate the problem of improving the performance of Artificial Intelligence systems in understanding document images, especially in cases where training data is limited. We address the problem by proposing a novel finetuning method using reinforcement learning. Our approach treats the Information Extraction model as a policy network and uses policy gradient training to update the model to maximize combined reward functions that complement the traditional cross-entropy losses. Our experiments on four datasets using labels and expert feedback demonstrate that our finetuning mechanism consistently improves the performance of a state-of-the-art information extractor, especially in the small training data regime.