Deep Learning for automatic sale receipt understanding

TL;DR

This paper presents a deep learning-based method for automatic understanding of sale receipts from smartphone images, combining neural networks and classical image/text processing to improve accuracy despite scanning imperfections.

Contribution

It introduces a double check processing pipeline that jointly uses DCNNs and traditional methods for more reliable receipt information extraction.

Findings

High confidence in store, products, and prices extraction achieved.

Robustness to scanning quality variations demonstrated.

Joint neural and classical processing improves accuracy.

Abstract

As a general rule, data analytics are now mandatory for companies. Scanned document analysis brings additional challenges introduced by paper damages and scanning quality.In an industrial context, this work focuses on the automatic understanding of sale receipts which enable access to essential and accurate consumption statistics. Given an image acquired with a smart-phone, the proposed work mainly focuses on the first steps of the full tool chain which aims at providing essential information such as the store brand, purchased products and related prices with the highest possible confidence. To get this high confidence level, even if scanning is not perfectly controlled, we propose a double check processing tool-chain using Deep Convolutional Neural Networks (DCNNs) on one hand and more classical image and text processings on another hand.The originality of this work relates in this double check processing and in the joint use of DCNNs for different applications and text analysis.