A Multimodal Assistive System for Helping Visually Impaired in Social Interactions
M. Saquib Sarfraz, Angela Constantinescu, Melanie Zuzej, Rainer, Stiefelhagen

TL;DR
This paper presents a real-time multimodal assistive system that provides non-verbal social cues to visually impaired users, demonstrating its usefulness and usability concerns through a user study.
Contribution
It introduces a novel multimodal system for social cues and evaluates its impact on visually impaired users in real social interactions.
Findings
Users find the system useful in social interactions.
Usability concerns need further addressing.
Provides insights for future assistive technology development.
Abstract
Access to non-verbal cues in social interactions is vital for people with visual impairment. It has been shown that non-verbal cues such as eye contact, number of people, their names and positions are helpful for individuals who are blind. While there is an increasing interest in developing systems to provide these cues less emphasis has been put in evaluating its impact on the visually impaired users. In this paper, we provide this analysis by conducting a user study with 12 visually impaired participants in a typical social interaction setting. We design a real time multi-modal system that provides such non-verbal cues via audio and haptic interfaces. The study shows that such systems are generally perceived as useful in social interaction and brings forward some concerns that are not being addressed in its usability aspects. The study provides important insight about developing such…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
