Privacy Preserving Personal Assistant with On-Device Diarization and Spoken Dialogue System for Home and Beyond

TL;DR

This paper presents a privacy-preserving personal assistant that uses on-device speech processing, speaker diarization, and sensor data fusion to enable personalized, secure, and context-aware conversations for home and elderly care.

Contribution

It introduces a novel on-device diarization and sensor data fusion approach to enhance privacy and personalization in voice assistants for home and healthcare applications.

Findings

On-device processing reduces privacy risks.

Sensor data fusion improves contextual understanding.

Personalized dialogue enhances user experience.

Abstract

In the age of personal voice assistants, the question of privacy arises. These digital companions often lack memory of past interactions, while relying heavily on the internet for speech processing, raising privacy concerns. Modern smartphones now enable on-device speech processing, making cloud-based solutions unnecessary. Personal assistants for the elderly should excel at memory recall, especially in medical examinations. The e-ViTA project developed a versatile conversational application with local processing and speaker recognition. This paper highlights the importance of speaker diarization enriched with sensor data fusion for contextualized conversation preservation. The use cases applied to the e-VITA project have shown that truly personalized dialogue is pivotal for individual voice assistants. Secure local processing and sensor data fusion ensure virtual companions meet individual user needs without compromising privacy or data security.