TILT: Transform Invariant Low-rank Textures

TL;DR

This paper introduces a method to extract low-rank textures from 2D images of 3D scenes, effectively handling corruptions and deformations, enabling accurate recovery of geometric and appearance information for regular patterns and objects.

Contribution

The paper presents a novel convex optimization-based approach to robustly recover low-rank textures and their transformations under affine or projective deformations.

Findings

Effective recovery of low-rank textures in various patterns

Accurate estimation of domain transformations and 3D geometry

Works well with regular, symmetric, and near-regular patterns

Abstract

In this paper, we show how to efficiently and effectively extract a class of "low-rank textures" in a 3D scene from 2D images despite significant corruptions and warping. The low-rank textures capture geometrically meaningful structures in an image, which encompass conventional local features such as edges and corners as well as all kinds of regular, symmetric patterns ubiquitous in urban environments and man-made objects. Our approach to finding these low-rank textures leverages the recent breakthroughs in convex optimization that enable robust recovery of a high-dimensional low-rank matrix despite gross sparse errors. In the case of planar regions with significant affine or projective deformation, our method can accurately recover both the intrinsic low-rank texture and the precise domain transformation, and hence the 3D geometry and appearance of the planar regions. Extensive experimental results demonstrate that this new technique works effectively for many regular and near-regular patterns or objects that are approximately low-rank, such as symmetrical patterns, building facades, printed texts, and human faces.