MTPareto: A MultiModal Targeted Pareto Framework for Fake News Detection

TL;DR

The paper introduces MTPareto, a novel multimodal fusion framework using Targeted Pareto optimization to improve fake news detection accuracy by effectively balancing multiple objectives at different fusion levels.

Contribution

It proposes a hierarchical fusion network with a Targeted Pareto optimization algorithm for multimodal fake news detection, addressing optimization conflicts in bimodal fusion.

Findings

Outperforms baseline methods on FakeSV and FVC datasets.

Achieves 2.40% and 1.89% accuracy improvements.

Demonstrates effective fusion-level-specific objective learning.

Abstract

Multimodal fake news detection is essential for maintaining the authenticity of Internet multimedia information. Significant differences in form and content of multimodal information lead to intensified optimization conflicts, hindering effective model training as well as reducing the effectiveness of existing fusion methods for bimodal. To address this problem, we propose the MTPareto framework to optimize multimodal fusion, using a Targeted Pareto(TPareto) optimization algorithm for fusion-level-specific objective learning with a certain focus. Based on the designed hierarchical fusion network, the algorithm defines three fusion levels with corresponding losses and implements all-modal-oriented Pareto gradient integration for each. This approach accomplishes superior multimodal fusion by utilizing the information obtained from intermediate fusion to provide positive effects to the entire process. Experiment results on FakeSV and FVC datasets show that the proposed framework outperforms baselines and the TPareto optimization algorithm achieves 2.40% and 1.89% accuracy improvement respectively.