Optimising Different Feature Types for Inpainting-based Image Representations

TL;DR

This paper presents a unified, automatic method for optimizing various feature types and their locations in inpainting-based image compression, significantly improving performance over traditional color data approaches.

Contribution

It introduces a generalised inpainting framework capable of handling arbitrary features and an algorithm to optimize feature selection, location, and values simultaneously.

Findings

Outperforms traditional color data inpainting at the same feature density

Handles arbitrary features expressed as linear constraints

Demonstrates effectiveness with a new feature set including local averages

Abstract

Inpainting-based image compression is a promising alternative to classical transform-based lossy codecs. Typically it stores a carefully selected subset of all pixel locations and their colour values. In the decoding phase the missing information is reconstructed by an inpainting process such as homogeneous diffusion inpainting. Optimising the stored data is the key for achieving good performance. A few heuristic approaches also advocate alternative feature types such as derivative data and construct dedicated inpainting concepts. However, one still lacks a general approach that allows to optimise and inpaint the data simultaneously w.r.t. a collection of different feature types, their locations, and their values. Our paper closes this gap. We introduce a generalised inpainting process that can handle arbitrary features which can be expressed as linear equality constraints. This includes e.g. colour values and derivatives of any order. We propose a fully automatic algorithm that aims at finding the optimal features from a given collection as well as their locations and their function values within a specified total feature density. Its performance is demonstrated with a novel set of features that also includes local averages. Our experiments show that it clearly outperforms the popular inpainting with optimised colour data with the same density.