Processing of incomplete images by (graph) convolutional neural networks

TL;DR

This paper introduces a graph-based neural network approach for processing incomplete images without imputing missing pixels, outperforming traditional CNNs with imputation in classification and reconstruction tasks.

Contribution

It proposes using spatial graph convolutional networks to directly handle incomplete images, avoiding data imputation and establishing a natural link to classical CNNs.

Findings

SGCN outperforms CNNs with imputed data in experiments

The approach effectively handles missing pixels in image processing

It improves classification and reconstruction accuracy

Abstract

We investigate the problem of training neural networks from incomplete images without replacing missing values. For this purpose, we first represent an image as a graph, in which missing pixels are entirely ignored. The graph image representation is processed using a spatial graph convolutional network (SGCN) -- a type of graph convolutional networks, which is a proper generalization of classical CNNs operating on images. On one hand, our approach avoids the problem of missing data imputation while, on the other hand, there is a natural correspondence between CNNs and SGCN. Experiments confirm that our approach performs better than analogical CNNs with the imputation of missing values on typical classification and reconstruction tasks.