Semantic Human Parsing via Scalable Semantic Transfer over Multiple Label Domains

TL;DR

This paper introduces Scalable Semantic Transfer (SST), a training paradigm that leverages multiple label domains to improve human parsing models, enabling flexible predictions and knowledge transfer without extra inference costs.

Contribution

The paper proposes SST, a novel, extensible training framework for human parsing that transfers semantic knowledge across label domains and supports both universal and dedicated parsing.

Findings

SST achieves promising universal human parsing performance.

It significantly improves accuracy on three benchmarks.

Modules are auxiliary and removed during inference, saving costs.

Abstract

This paper presents Scalable Semantic Transfer (SST), a novel training paradigm, to explore how to leverage the mutual benefits of the data from different label domains (i.e. various levels of label granularity) to train a powerful human parsing network. In practice, two common application scenarios are addressed, termed universal parsing and dedicated parsing, where the former aims to learn homogeneous human representations from multiple label domains and switch predictions by only using different segmentation heads, and the latter aims to learn a specific domain prediction while distilling the semantic knowledge from other domains. The proposed SST has the following appealing benefits: (1) it can capably serve as an effective training scheme to embed semantic associations of human body parts from multiple label domains into the human representation learning process; (2) it is an extensible semantic transfer framework without predetermining the overall relations of multiple label domains, which allows continuously adding human parsing datasets to promote the training. (3) the relevant modules are only used for auxiliary training and can be removed during inference, eliminating the extra reasoning cost. Experimental results demonstrate SST can effectively achieve promising universal human parsing performance as well as impressive improvements compared to its counterparts on three human parsing benchmarks (i.e., PASCAL-Person-Part, ATR, and CIHP). Code is available at https://github.com/yangjie-cv/SST.