MCA: 2D-3D Retrieval with Noisy Labels via Multi-level Adaptive Correction and Alignment

TL;DR

This paper introduces MCA, a robust framework for 2D-3D cross-modal retrieval that effectively handles noisy labels through multi-level adaptive correction and alignment, improving accuracy on noisy benchmarks.

Contribution

The paper proposes a novel multi-level adaptive correction and alignment framework with joint label correction, enhancing robustness against noisy annotations in 2D-3D retrieval tasks.

Findings

Achieves state-of-the-art results on noisy 3D benchmarks.

Demonstrates robustness to label noise in cross-modal retrieval.

Outperforms existing methods in noisy label scenarios.

Abstract

With the increasing availability of 2D and 3D data, significant advancements have been made in the field of cross-modal retrieval. Nevertheless, the existence of imperfect annotations presents considerable challenges, demanding robust solutions for 2D-3D cross-modal retrieval in the presence of noisy label conditions. Existing methods generally address the issue of noise by dividing samples independently within each modality, making them susceptible to overfitting on corrupted labels. To address these issues, we propose a robust 2D-3D \textbf{M}ulti-level cross-modal adaptive \textbf{C}orrection and \textbf{A}lignment framework (MCA). Specifically, we introduce a Multimodal Joint label Correction (MJC) mechanism that leverages multimodal historical self-predictions to jointly model the modality prediction consistency, enabling reliable label refinement. Additionally, we propose a Multi-level Adaptive Alignment (MAA) strategy to effectively enhance cross-modal feature semantics and discrimination across different levels. Extensive experiments demonstrate the superiority of our method, MCA, which achieves state-of-the-art performance on both conventional and realistic noisy 3D benchmarks, highlighting its generality and effectiveness.