From the Token to the Review: A Hierarchical Multimodal approach to Opinion Mining

TL;DR

This paper introduces a hierarchical multimodal model for fine-grained opinion mining from spontaneous spoken language, bridging the gap between written and spoken opinion models using a joint approach.

Contribution

It presents a novel hierarchical multimodal approach that combines fine and coarse grained opinion modeling for spontaneous spoken language, leveraging implicit opinion structures.

Findings

Achieves competitive results on a multimodal annotated corpus.

Effectively combines multiple views of opinion expression.

Bridges the gap between written and spoken opinion models.

Abstract

The task of predicting fine grained user opinion based on spontaneous spoken language is a key problem arising in the development of Computational Agents as well as in the development of social network based opinion miners. Unfortunately, gathering reliable data on which a model can be trained is notoriously difficult and existing works rely only on coarsely labeled opinions. In this work we aim at bridging the gap separating fine grained opinion models already developed for written language and coarse grained models developed for spontaneous multimodal opinion mining. We take advantage of the implicit hierarchical structure of opinions to build a joint fine and coarse grained opinion model that exploits different views of the opinion expression. The resulting model shares some properties with attention-based models and is shown to provide competitive results on a recently released multimodal fine grained annotated corpus.