Analysis of Temporal Features for Interaction Quality Estimation

TL;DR

This paper investigates the impact of temporal features on estimating interaction quality in spoken dialogue systems, showing that including temporal features and optimizing window size significantly improves classification accuracy.

Contribution

It extends the set of temporal features to include system and user views and demonstrates the importance of window size in modeling temporal effects for IQ estimation.

Findings

Temporal features significantly improve IQ classification.

Including system and user views enhances feature set.

Optimizing window size increases performance by 15.69%.

Abstract

Many different approaches for estimating the Interaction Quality (IQ) of Spoken Dialogue Systems have been investigated. While dialogues clearly have a sequential nature, statistical classification approaches designed for sequential problems do not seem to work better on automatic IQ estimation than static approaches, i.e., regarding each turn as being independent of the corresponding dialogue. Hence, we analyse this effect by investigating the subset of temporal features used as input for statistical classification of IQ. We extend the set of temporal features to contain the system and the user view. We determine the contribution of each feature sub-group showing that temporal features contribute most to the classification performance. Furthermore, for the feature sub-group modeling the temporal effects with a window, we modify the window size increasing the overall performance significantly by +15.69%.