Pay Voice: Point of Sale Recognition for Visually Impaired People

TL;DR

Pay Voice is a smartphone app that helps visually impaired users verify payment details on POS and PIN pad machines using image processing, OCR, and voice feedback, achieving over 80% accuracy in real-world tests.

Contribution

This work introduces a novel mobile application combining image processing, OCR, and voice synthesis to assist visually impaired individuals in verifying payment machine information.

Findings

Over 80% accuracy in real-world scenarios

Recognition processing time less than 5 seconds

App is publicly available on Google Play and App Store

Abstract

Millions of visually impaired people depend on relatives and friends to perform their everyday tasks. One relevant step towards self-sufficiency is to provide them with means to verify the value and operation presented in payment machines. In this work, we developed and released a smartphone application, named Pay Voice, that uses image processing, optical character recognition (OCR) and voice synthesis to recognize the value and operation presented in POS and PIN pad machines, and thus informing the user with auditive and visual feedback. The proposed approach presented significant results for value and operation recognition, especially for POS, due to the higher display quality. Importantly, we achieved the key performance indicators, namely, more than 80% of accuracy in a real-world scenario, and less than $5$ seconds of processing time for recognition. Pay Voice is publicly available on Google Play and App Store for free.