# TSFF: a two-stage fusion framework for 3D object detection

## TL;DR

This paper introduces a two-stage fusion framework to improve 3D object detection by combining image and point cloud data.

## Contribution

The novel TSFF framework enhances 3D object detection by fusing image and point cloud data in two stages.

## Key findings

- TSFF achieved a 3.6 mAP improvement on the SUNRGB-D dataset compared to the baseline.
- The method performed well in detecting objects in sparse scenes with occlusions.
- The constrained fusion module effectively reduced background point interference.

## Abstract

Point clouds are highly regarded in the field of 3D object detection for their superior geometric properties and versatility. However, object occlusion and defects in scanning equipment frequently result in sparse and missing data within point clouds, adversely affecting the final prediction. Recognizing the synergistic potential between the rich semantic information present in images and the geometric data in point clouds for scene representation, we introduce a two-stage fusion framework (TSFF) for 3D object detection. To address the issue of corrupted geometric information in point clouds caused by object occlusion, we augment point features with image features, thereby enhancing the reference factor of the point cloud during the voting bias phase. Furthermore, we implement a constrained fusion module to selectively sample voting points using a 2D bounding box, integrating valuable image features while reducing the impact of background points in sparse scenes. Our methodology was evaluated on the SUNRGB-D dataset, where it achieved a 3.6 mean average percent (mAP) improvement in the mAP@0.25 evaluation criterion over the baseline. In comparison to other great 3D object detection methods, our method had excellent performance in the detection of some objects.