Deeply Matting-based Dual Generative Adversarial Network for Image and Document Label Supervision

TL;DR

This paper introduces mdGAN, a novel deep learning framework that enhances document image super-resolution by decomposing images into layers and using text labels for improved restoration accuracy.

Contribution

The proposed mdGAN uniquely combines image matting with dual GAN branches and supervised text labels to improve document image super-resolution performance.

Findings

Outperforms state-of-the-art methods quantitatively.

Achieves better qualitative restoration of text images.

Effectively decomposes images into text, foreground, background layers.

Abstract

Although many methods have been proposed to deal with nature image super-resolution (SR) and get impressive performance, the text images SR is not good due to their ignorance of document images. In this paper, we propose a matting-based dual generative adversarial network (mdGAN) for document image SR. Firstly, the input image is decomposed into document text, foreground and background layers using deep image matting. Then two parallel branches are constructed to recover text boundary information and color information respectively. Furthermore, in order to improve the restoration accuracy of characters in output image, we use the input image's corresponding ground truth text label as extra supervise information to refine the two-branch networks during training. Experiments on real text images demonstrate that our method outperforms several state-of-the-art methods quantitatively and qualitatively.