Information-Flow Matting

TL;DR

This paper introduces a new affinity-based natural image matting algorithm that effectively models pixel relationships to improve matting quality, robustness, and layer color estimation, with applications in green-screen keying.

Contribution

It proposes a novel linear system formulation using multiple pixel affinity definitions, including color-mixture flow, for improved natural image matting.

Findings

Robust against holes and intricate structures in images.

Enhances layer color quality through multi-channel flow.

Effective in green-screen keying applications.

Abstract

We present a novel, purely affinity-based natural image matting algorithm. Our method relies on carefully defined pixel-to-pixel connections that enable effective use of information available in the image. We control the information flow from the known-opacity regions into the unknown region, as well as within the unknown region itself, by utilizing multiple definitions of pixel affinities. Among other forms of information flow, we introduce color-mixture flow, which builds upon local linear embedding and effectively encapsulates the relation between different pixel opacities. Our resulting novel linear system formulation can be solved in closed-form and is robust against several fundamental challenges of natural matting such as holes and remote intricate structures. While our method is primarily designed as a standalone matting tool, we show that it can also be used for regularizing mattes obtained by sampling-based methods. The formulation is also extended to layer color estimation and we show that the use of multiple channels of flow increases the layer color quality. We also demonstrate our performance in green-screen keying and analyze the characteristics of the utilized affinities.