Class-weighted Dempster–Shafer in dual-level fusion for multimodal fake real estate listings detection

TL;DR

This paper introduces a new method for detecting fake real estate listings by combining text and image data with a weighted decision fusion technique.

Contribution

The novel contribution is the Class-weighted Dempster–Shafer in Dual Level Fusion (CWDS-DLF) method for multimodal fraud detection.

Findings

CWDS-DLF achieved an F1 score of 96% and accuracy of 93% in detecting fake property listings.

The method outperformed other models like CNN and XGBoost in precision, recall, and F1-score.

A t-test confirmed the statistical significance of the improvements (p < 0.05).

Abstract

Detecting fake multimodal property listings is a significant challenge in online real estate platforms due to the increasing sophistication of fraudulent activities. The existing multimodal data fusion methods have several limitations and strengths in identifying fraudulent listings. Single-level fusion models whether at the feature, decision, or intermediate level struggle with balancing the contributions of different modalities leading to suboptimal decision-making. To address these problems, a dual-level fusion from multimodal for fake real estate listings detection is proposed. The dual-level fusion allows the integration of detailed features from text and image data to be performed at an early stage, followed by the metadata fusion at the decision stage in order to obtain a more comprehensive final classification. Furthermore, a new weighting scheme is introduced to optimize Dempster–Shafer in decision fusion to help the model achieve optimal performance and as a result, our method improves the classification. The Dempster–Shafer without class weightage lacks the flexibility to adapt to varying levels of uncertainty or importance across different classes.

In Class Weighted Dempster–Shafer in Dual Level Fusion (CWDS-DLF), we employ advanced models (XLNet for text and ResNet101 for images) for feature extraction and use the Dempster–Shafer theory for decision fusion. A new weighting scheme, based on Bayesian optimization, was used to assign optimal weights to the ‘fake’ and ‘not fake’ classes, thereby enhancing the Dempster–Shafer theory in the decision fusion process.

The CWDS-DLF was evaluated on the property listing website dataset and achieved an F1 score of 96% and an accuracy of 93%. A t-test confirms the significance of these improvements (p < 0.05), demonstrating the effectiveness of our method in detecting fake property listings. Compared to other models, including 2D-convolutional neural network (CNN), XGBoost, and various multimodal approaches, our model consistently outperforms in precision, recall, and F1-score. This underscores the potential of integrating multimodal analysis with sophisticated fusion techniques to enhance the detection of fake property listings, ultimately improving consumer protection and operational efficiency in online real estate platforms.

Figures