Multimodal Fine-grained Reasoning for Post Quality Evaluation

TL;DR

This paper introduces MFTRR, a multimodal framework for fine-grained post quality evaluation that models complex semantic and relational cues, significantly outperforming existing unimodal methods.

Contribution

The paper proposes a novel multimodal reasoning framework with modules for semantic correlation and evidential relational reasoning, addressing limitations of previous unimodal approaches.

Findings

MFTRR achieves up to 9.52% NDCG@3 improvement over baselines.

It effectively models fine-grained semantic interactions at multiple levels.

Experimental results on three datasets validate its superior performance.

Abstract

Accurately assessing post quality requires complex relational reasoning to capture nuanced topic-post relationships. However, existing studies face three major limitations: (1) treating the task as unimodal categorization, which fails to leverage multimodal cues and fine-grained quality distinctions; (2) introducing noise during deep multimodal fusion, leading to misleading signals; and (3) lacking the ability to capture complex semantic relationships like relevance and comprehensiveness. To address these issues, we propose the Multimodal Fine-grained Topic-post Relational Reasoning (MFTRR) framework, which mimics human cognitive processes. MFTRR reframes post-quality assessment as a ranking task and incorporates multimodal data to better capture quality variations. It consists of two key modules: (1) the Local-Global Semantic Correlation Reasoning Module, which models fine-grained semantic interactions between posts and topics at both local and global levels, enhanced by a maximum information fusion mechanism to suppress noise; and (2) the Multi-Level Evidential Relational Reasoning Module, which explores macro- and micro-level relational cues to strengthen evidence-based reasoning. We evaluate MFTRR on three newly constructed multimodal topic-post datasets and the public Lazada-Home dataset. Experimental results demonstrate that MFTRR significantly outperforms state-of-the-art baselines, achieving up to 9.52% NDCG@3 improvement over the best unimodal method on the Art History dataset.