Effectiveness of Function Matching in Driving Scene Recognition

TL;DR

This paper demonstrates that using large-scale unlabeled data for function matching in knowledge distillation significantly enhances the performance of compact models in driving scene recognition, matching large teacher models.

Contribution

It introduces the application of function matching with massive unlabeled data to improve autonomous driving scene recognition models.

Findings

Student models' performance improves dramatically with unlabeled data.

Knowledge distillation with large unlabeled datasets can match large teacher models.

Extensive experiments validate the effectiveness of the approach.

Abstract

Knowledge distillation is an effective approach for training compact recognizers required in autonomous driving. Recent studies on image classification have shown that matching student and teacher on a wide range of data points is critical for improving performance in distillation. This concept (called function matching) is suitable for driving scene recognition, where generally an almost infinite amount of unlabeled data are available. In this study, we experimentally investigate the impact of using such a large amount of unlabeled data for distillation on the performance of student models in structured prediction tasks for autonomous driving. Through extensive experiments, we demonstrate that the performance of the compact student model can be improved dramatically and even match the performance of the large-scale teacher by knowledge distillation with massive unlabeled data.