Machines Serve Human: A Novel Variable Human-machine Collaborative Compression Framework

TL;DR

This paper introduces a novel collaborative compression framework where machine vision guides human vision compression, utilizing a diffusion prior to enhance detail restoration, resulting in superior performance over existing methods.

Contribution

It presents the first machine-vision-oriented collaborative compression method, with a plug-and-play variable bit-rate strategy and diffusion-prior based feature aggregation for improved image/video compression.

Findings

Superior performance on machine-vision compression tasks

Enhanced human-vision image/video quality

Effective semantic aggregation from machine to human vision

Abstract

Human-machine collaborative compression has been receiving increasing research efforts for reducing image/video data, serving as the basis for both human perception and machine intelligence. Existing collaborative methods are dominantly built upon the de facto human-vision compression pipeline, witnessing deficiency on complexity and bit-rates when aggregating the machine-vision compression. Indeed, machine vision solely focuses on the core regions within the image/video, requiring much less information compared with the compressed information for human vision. In this paper, we thus set out the first successful attempt by a novel collaborative compression method based on the machine-vision-oriented compression, instead of human-vision pipeline. In other words, machine vision serves as the basis for human vision within collaborative compression. A plug-and-play variable bit-rate strategy is also developed for machine vision tasks. Then, we propose to progressively aggregate the semantics from the machine-vision compression, whilst seamlessly tailing the diffusion prior to restore high-fidelity details for human vision, thus named as diffusion-prior based feature compression for human and machine visions (Diff-FCHM). Experimental results verify the consistently superior performances of our Diff-FCHM, on both machine-vision and human-vision compression with remarkable margins. Our code will be released upon acceptance.