Multi-Task Learning for Robot Perception with Imbalanced Data

TL;DR

This paper introduces a multi-task learning method for robot perception that effectively handles imbalanced data and missing labels, improving task performance with limited data and analyzing task interactions.

Contribution

It proposes a novel approach enabling learning with incomplete labels and analyzes task interactions to enhance multi-task learning for robot perception.

Findings

Method effectively learns with missing labels.

Task interactions can improve overall performance.

Empirical results on NYUDv2 and Cityscapes datasets.

Abstract

Multi-task problem solving has been shown to improve the accuracy of the individual tasks, which is an important feature for robots, as they have a limited resource. However, when the number of labels for each task is not equal, namely imbalanced data exist, a problem may arise due to insufficient number of samples, and labeling is not very easy for mobile robots in every environment. We propose a method that can learn tasks even in the absence of the ground truth labels for some of the tasks. We also provide a detailed analysis of the proposed method. An interesting finding is related to the interaction of the tasks. We show a methodology to find out which tasks can improve the performance of other tasks. We investigate this by training the teacher network with the task outputs such as depth as inputs. We further provide empirical evidence when trained with a small amount of data. We use semantic segmentation and depth estimation tasks on different datasets, NYUDv2 and Cityscapes.