Optical Font Recognition in Smartphone-Captured Images, and its Applicability for ID Forgery Detection

TL;DR

This paper explores using convolutional neural networks with multi-task learning to detect counterfeit fonts in smartphone-captured ID images, aiding forgery detection with high accuracy and generalization.

Contribution

It introduces a multi-task CNN approach for font authentication in ID documents, improving detection sensitivity and generalization over previous methods.

Findings

Multi-task learning improves classifier sensitivity and specificity.

CNNs generalize well to unseen fonts.

Method is effective for smartphone-acquired ID image forgery detection.

Abstract

In this paper, we consider the problem of detecting counterfeit identity documents in images captured with smartphones. As the number of documents contain special fonts, we study the applicability of convolutional neural networks (CNNs) for detection of the conformance of the fonts used with the ones, corresponding to the government standards. Here, we use multi-task learning to differentiate samples by both fonts and characters and compare the resulting classifier with its analogue trained for binary font classification. We train neural networks for authenticity estimation of the fonts used in machine-readable zones and ID numbers of the Russian national passport and test them on samples of individual characters acquired from 3238 images of the Russian national passport. Our results show that the usage of multi-task learning increases sensitivity and specificity of the classifier. Moreover, the resulting CNNs demonstrate high generalization ability as they correctly classify fonts which were not present in the training set. We conclude that the proposed method is sufficient for authentication of the fonts and can be used as a part of the forgery detection system for images acquired with a smartphone camera.