Online Graph Completion: Multivariate Signal Recovery in Computer Vision

TL;DR

This paper introduces a graph-based approach for sequentially completing missing data in computer vision tasks, leveraging adaptive submodularity and Fourier domain optimization, with promising results on image datasets and neuroimaging applications.

Contribution

It develops a novel graph completion framework for sequential data acquisition in vision tasks, integrating adaptive submodularity and Fourier domain methods.

Findings

Effective completion of image datasets with partial measurements

Promising results in neuroimaging experimental design

Algorithms perform well in practical vision scenarios

Abstract

The adoption of "human-in-the-loop" paradigms in computer vision and machine learning is leading to various applications where the actual data acquisition (e.g., human supervision) and the underlying inference algorithms are closely interwined. While classical work in active learning provides effective solutions when the learning module involves classification and regression tasks, many practical issues such as partially observed measurements, financial constraints and even additional distributional or structural aspects of the data typically fall outside the scope of this treatment. For instance, with sequential acquisition of partial measurements of data that manifest as a matrix (or tensor), novel strategies for completion (or collaborative filtering) of the remaining entries have only been studied recently. Motivated by vision problems where we seek to annotate a large dataset of images via a crowdsourced platform or alternatively, complement results from a state-of-the-art object detector using human feedback, we study the "completion" problem defined on graphs, where requests for additional measurements must be made sequentially. We design the optimization model in the Fourier domain of the graph describing how ideas based on adaptive submodularity provide algorithms that work well in practice. On a large set of images collected from Imgur, we see promising results on images that are otherwise difficult to categorize. We also show applications to an experimental design problem in neuroimaging.