Audiovisual Affect Assessment and Autonomous Automobiles: Applications

TL;DR

This paper explores the integration of audiovisual emotion recognition in autonomous vehicles to monitor passenger wellbeing, adapt driving and entertainment, and discusses technical challenges and practical applications.

Contribution

It proposes a comprehensive framework for affect modeling in autonomous cars, including passenger monitoring and diarisation, highlighting recent advancements and future challenges.

Findings

Affect analysis is now applicable in autonomous vehicles.

Holistic passenger modeling can improve safety and comfort.

Automated affect recognition is ready for real-world use cases.

Abstract

Emotion and a broader range of affective driver states can be a life decisive factor on the road. While this aspect has been investigated repeatedly, the advent of autonomous automobiles puts a new perspective on the role of computer-based emotion recognition in the car -- the passenger's one. This includes amongst others the monitoring of wellbeing during the commute such as to adjust the driving style or to adapt the info- and entertainment. This contribution aims to foresee according challenges and provide potential avenues towards affect modelling in a multimodal "audiovisual plus x" on the road context. From the technical end, this concerns holistic passenger modelling and reliable diarisation of the individuals in a vehicle. In conclusion, automated affect analysis has just matured to the point of applicability in autonomous vehicles in first selected use-cases, which will be discussed towards the end.