Object Recognition System on a Tactile Device for Visually Impaired

TL;DR

This paper presents a tactile device integrated with an object recognition system on a Raspberry Pi, converting visual scenes into auditory feedback to assist visually impaired users in understanding their environment.

Contribution

It introduces a novel tactile-auditory interface combined with an optimized object detection model for real-time scene understanding on low-cost hardware.

Findings

Effective recognition of static and dynamic objects

Successful identification of screen contents like TVs and phones

Device improves environmental awareness for visually impaired users

Abstract

People with visual impairments face numerous challenges when interacting with their environment. Our objective is to develop a device that facilitates communication between individuals with visual impairments and their surroundings. The device will convert visual information into auditory feedback, enabling users to understand their environment in a way that suits their sensory needs. Initially, an object detection model is selected from existing machine learning models based on its accuracy and cost considerations, including time and power consumption. The chosen model is then implemented on a Raspberry Pi, which is connected to a specifically designed tactile device. When the device is touched at a specific position, it provides an audio signal that communicates the identification of the object present in the scene at that corresponding position to the visually impaired individual. Conducted tests have demonstrated the effectiveness of this device in scene understanding, encompassing static or dynamic objects, as well as screen contents such as TVs, computers, and mobile phones.