Physics-Guided Fusion for Robust 3D Tracking of Fast Moving Small Objects

TL;DR

This paper presents a physics-guided fusion system that enhances 3D tracking of fast-moving small objects using RGB-D data, combining deep learning detection with physics-based tracking for improved accuracy in challenging scenarios.

Contribution

It introduces a novel system integrating deep learning and physics-based models for robust 3D tracking of small, fast-moving objects, outperforming traditional methods.

Findings

Achieved up to 70% reduction in Average Displacement Error compared to Kalman filter trackers.

Demonstrated effective tracking during occlusions and rapid direction changes.

Validated system performance on a custom racquetball dataset.

Abstract

While computer vision has advanced considerably for general object detection and tracking, the specific problem of fast-moving tiny objects remains underexplored. This paper addresses the significant challenge of detecting and tracking rapidly moving small objects using an RGB-D camera. Our novel system combines deep learning-based detection with physics-based tracking to overcome the limitations of existing approaches. Our contributions include: (1) a comprehensive system design for object detection and tracking of fast-moving small objects in 3D space, (2) an innovative physics-based tracking algorithm that integrates kinematics motion equations to handle outliers and missed detections, and (3) an outlier detection and correction module that significantly improves tracking performance in challenging scenarios such as occlusions and rapid direction changes. We evaluated our proposed system on a custom racquetball dataset. Our evaluation shows our system surpassing kalman filter based trackers with up to 70\% less Average Displacement Error. Our system has significant applications for improving robot perception on autonomous platforms and demonstrates the effectiveness of combining physics-based models with deep learning approaches for real-time 3D detection and tracking of challenging small objects.