Laughter Synthesis: Combining Seq2seq modeling with Transfer Learning

TL;DR

This paper introduces a novel laughter synthesis system using sequence-to-sequence TTS models and transfer learning, achieving higher naturalness than traditional methods and enabling speech synthesis with controllable amusement levels.

Contribution

It presents the first integration of laughter synthesis into a seq2seq TTS system using transfer learning, enhancing naturalness and control over emotional expressions.

Findings

The proposed model outperforms HMM-based laughter synthesis in perceived naturalness.

Transfer learning effectively enables joint speech and laughter generation.

The system is a step toward emotionally expressive speech synthesis with laughter control.

Abstract

Despite the growing interest for expressive speech synthesis, synthesis of nonverbal expressions is an under-explored area. In this paper we propose an audio laughter synthesis system based on a sequence-to-sequence TTS synthesis system. We leverage transfer learning by training a deep learning model to learn to generate both speech and laughs from annotations. We evaluate our model with a listening test, comparing its performance to an HMM-based laughter synthesis one and assess that it reaches higher perceived naturalness. Our solution is a first step towards a TTS system that would be able to synthesize speech with a control on amusement level with laughter integration.