Improving Computer Assisted Speech Therapy Through Speech Based Emotion Recognition

TL;DR

This paper introduces PhonEM, an emotion recognition framework based on speech analysis, to enhance computer-assisted speech therapy by enabling more empathetic and responsive virtual therapists, especially for children.

Contribution

It presents a novel fuzzy model for emotion representation and a speech-based detection method integrated into a virtual speech therapy system, improving emotional responsiveness.

Findings

Successful integration of PhonEM into CBST Logomon.

Enhanced emotional responsiveness in speech therapy sessions.

Positive preliminary results encouraging further development.

Abstract

Speech therapy consists in a wide range of services whose aim is to prevent, diagnose and treat different types of speech impairments. One of the most important conditions for obtaining favourable and steady results is the "immersing" of the subject as long as possible into therapeutic context: at home, at school/work, on the street. Since nowadays portable computers tend to become habitual accessories, it seems a good idea to create virtual versions of human SLTs and to integrate them into these devices. However one of the main distinctions between a Speech and Language Therapist (SLT) and a Computer Based Speech Therapy System (CBST) arise from the field of emotion intelligence. The inability of current CBSTs to detect emotional state of human subjects leads to inadequate behavioural responses. Furthermore, this "unresponsive" behaviour is perceived as a lack of empathy and, especially when subjects are children, leads to negative emotional state such as frustration. Thus in this article we propose an original emotions recognition framework - named PhonEM - to be integrated in our previous developed CBST - Logomon. The originality consists in both emotions representation (a fuzzy model) and detection (using only subjects' speech stream). These exceptional restrictions along with the fuzzy representation of emotions lie at the origin of our approach and make our task a difficult and, in the same time, an innovative one. As far as we know, this is the first attempt to combine these techniques in order to improve assisted speech therapy and the obtained results encourage as to further develop our CBST.