Object Detection-Based Variable Quantization Processing

TL;DR

This paper introduces a content-aware preprocessing technique for image and video encoders that leverages object detection to adaptively optimize quantization, improving perceived quality without increasing output size.

Contribution

It presents a novel object detection-based adaptive quantization method that enhances traditional encoders by making them content-aware, applicable to JPEG and other DCT-based codecs.

Findings

Improved MS-SSIM at same bitrate

Enhanced viewing experience with perceptual quality

Applicable to JPEG, MPEG-2, H.264 encoders

Abstract

In this paper, we propose a preprocessing method for conventional image and video encoders that can make these existing encoders content-aware. By going through our process, a higher quality parameter could be set on a traditional encoder without increasing the output size. A still frame or an image will firstly go through an object detector. Either the properties of the detection result will decide the parameters of the following procedures, or the system will be bypassed if no object is detected in the given frame. The processing method utilizes an adaptive quantization process to determine the portion of data to be dropped. This method is primarily based on the JPEG compression theory and is optimum for JPEG-based encoders such as JPEG encoders and the Motion JPEG encoders. However, other DCT-based encoders like MPEG part 2, H.264, etc. can also benefit from this method. In the experiments, we compare the MS-SSIM under the same bitrate as well as similar MS-SSIM but enhanced bitrate. As this method is based on human perception, even with similar MS-SSIM, the overall watching experience will be better than the direct encoded ones.