Multimodal Quantum Natural Language Processing: A Novel Framework for using Quantum Methods to Analyse Real Data

TL;DR

This paper introduces a novel multimodal quantum natural language processing framework that leverages quantum computing to improve language understanding and image-text classification, demonstrating the effectiveness of syntax-based models.

Contribution

It advances MQNLP by applying quantum methods and comparing four compositional models, highlighting the superiority of syntax-based approaches in multimodal language tasks.

Findings

Syntax-based models outperform others in capturing grammatical structures.

Quantum methods show promise in enhancing language modeling.

DisCoCat and TreeReader excel in image-text classification tasks.

Abstract

Despite significant advances in quantum computing across various domains, research on applying quantum approaches to language compositionality - such as modeling linguistic structures and interactions - remains limited. This gap extends to the integration of quantum language data with real-world data from sources like images, video, and audio. This thesis explores how quantum computational methods can enhance the compositional modeling of language through multimodal data integration. Specifically, it advances Multimodal Quantum Natural Language Processing (MQNLP) by applying the Lambeq toolkit to conduct a comparative analysis of four compositional models and evaluate their influence on image-text classification tasks. Results indicate that syntax-based models, particularly DisCoCat and TreeReader, excel in effectively capturing grammatical structures, while bag-of-words and sequential models struggle due to limited syntactic awareness. These findings underscore the potential of quantum methods to enhance language modeling and drive breakthroughs as quantum technology evolves.