Making DensePose fast and light

TL;DR

This paper presents a redesigned, lightweight DensePose R-CNN model that maintains accuracy while significantly reducing size and inference latency, enabling deployment on mobile and embedded devices.

Contribution

The authors redesigned the DensePose R-CNN architecture using recent deep learning innovations, achieving substantial size reduction and speedup without sacrificing accuracy.

Findings

17x model size reduction

2x latency improvement

Effective use of efficient backbones and quantization

Abstract

DensePose estimation task is a significant step forward for enhancing user experience computer vision applications ranging from augmented reality to cloth fitting. Existing neural network models capable of solving this task are heavily parameterized and a long way from being transferred to an embedded or mobile device. To enable Dense Pose inference on the end device with current models, one needs to support an expensive server-side infrastructure and have a stable internet connection. To make things worse, mobile and embedded devices do not always have a powerful GPU inside. In this work, we target the problem of redesigning the DensePose R-CNN model's architecture so that the final network retains most of its accuracy but becomes more light-weight and fast. To achieve that, we tested and incorporated many deep learning innovations from recent years, specifically performing an ablation study on 23 efficient backbone architectures, multiple two-stage detection pipeline modifications, and custom model quantization methods. As a result, we achieved $17\times$ model size reduction and $2\times$ latency improvement compared to the baseline model.