Talking to Machines: do you read me?

TL;DR

This dissertation reviews the evolution of dialogue systems, highlighting research contributions in task-oriented and multimodal conversational agents, with a focus on deep learning, reinforcement learning, and large language models.

Contribution

It presents the author's research on task-oriented dialogues, supervised PhD work, and advancements in LLMs and multimodal dialogue systems, addressing open challenges in conversational AI.

Findings

Progress in modular and end-to-end dialogue architectures

Development of conversational QA systems

Integration of LLMs in task-oriented dialogue

Abstract

In this dissertation I would like to guide the reader to the research on dialogue but more precisely the research I have conducted during my career since my PhD thesis. Starting from modular architectures with machine learning/deep learning and reinforcement learning to end-to-end deep neural networks. Besides my work as research associate, I also present the work I have supervised in the last years. I review briefly the state of the art and highlight the open research problems on conversational agents. Afterwards, I present my contribution to Task-Oriented Dialogues (TOD), both as research associate and as the industrial supervisor of CIFRE theses. I discuss conversational QA. Particularly, I present the work of two PhD candidates Thibault Cordier and Sebastien Montella; as well as the work of the young researcher Quentin Brabant. Finally, I present the scientific project, where I discuss about Large Language Models (LLMs) for Task-Oriented Dialogue and Multimodal Task-Oriented Dialogue.