Body Segmentation Using Multi-task Learning

TL;DR

This paper introduces a novel multi-task learning model called SPD that jointly learns human body segmentation, keypoint-based skeleton estimation, and dense pose prediction to improve segmentation accuracy in computer vision tasks.

Contribution

The paper presents a new multi-task model that shares knowledge across three related tasks, enhancing human body segmentation performance over existing methods.

Findings

SPD achieves competitive segmentation results on LIP and ATR datasets.

Adding related tasks improves overall segmentation accuracy.

Comprehensive ablation studies validate the effectiveness of multi-task learning.

Abstract

Body segmentation is an important step in many computer vision problems involving human images and one of the key components that affects the performance of all downstream tasks. Several prior works have approached this problem using a multi-task model that exploits correlations between different tasks to improve segmentation performance. Based on the success of such solutions, we present in this paper a novel multi-task model for human segmentation/parsing that involves three tasks, i.e., (i) keypoint-based skeleton estimation, (ii) dense pose prediction, and (iii) human-body segmentation. The main idea behind the proposed Segmentation--Pose--DensePose model (or SPD for short) is to learn a better segmentation model by sharing knowledge across different, yet related tasks. SPD is based on a shared deep neural network backbone that branches off into three task-specific model heads and is learned using a multi-task optimization objective. The performance of the model is analysed through rigorous experiments on the LIP and ATR datasets and in comparison to a recent (state-of-the-art) multi-task body-segmentation model. Comprehensive ablation studies are also presented. Our experimental results show that the proposed multi-task (segmentation) model is highly competitive and that the introduction of additional tasks contributes towards a higher overall segmentation performance.