Data exploitation: multi-task learning of object detection and semantic segmentation on partially annotated data

TL;DR

This paper investigates multi-task learning of object detection and semantic segmentation using partially annotated data, proposing knowledge distillation to improve joint task performance and demonstrating its effectiveness through extensive experiments.

Contribution

It introduces a novel approach combining multi-task learning with knowledge distillation to handle partial annotations in object detection and segmentation.

Findings

Multi-task learning with knowledge distillation outperforms single-task models.

Joint optimization improves performance even with partial annotations.

The approach is effective compared to full supervision scenarios.

Abstract

Multi-task partially annotated data where each data point is annotated for only a single task are potentially helpful for data scarcity if a network can leverage the inter-task relationship. In this paper, we study the joint learning of object detection and semantic segmentation, the two most popular vision problems, from multi-task data with partial annotations. Extensive experiments are performed to evaluate each task performance and explore their complementarity when a multi-task network cannot optimize both tasks simultaneously. We propose employing knowledge distillation to leverage joint-task optimization. The experimental results show favorable results for multi-task learning and knowledge distillation over single-task learning and even full supervision scenario. All code and data splits are available at https://github.com/lhoangan/multas