Context-based Image Segment Labeling (CBISL)

TL;DR

This paper introduces CBISL, a novel context-based image segment labeling method using a quadro-directional PixelCNN to recover missing objects and their positions, outperforming previous models and achieving human-like accuracy.

Contribution

The paper presents a new quadro-directional PixelCNN approach for semantic image feature recovery, enhancing object and position prediction in incomplete images.

Findings

Quadro-directional PixelCNN outperforms one-directional models.

Achieves human-comparable performance in segment labeling.

Effective in recovering missing objects and their positions.

Abstract

Working with images, one often faces problems with incomplete or unclear information. Image inpainting can be used to restore missing image regions but focuses, however, on low-level image features such as pixel intensity, pixel gradient orientation, and color. This paper aims to recover semantic image features (objects and positions) in images. Based on published gated PixelCNNs, we demonstrate a new approach referred to as quadro-directional PixelCNN to recover missing objects and return probable positions for objects based on the context. We call this approach context-based image segment labeling (CBISL). The results suggest that our four-directional model outperforms one-directional models (gated PixelCNN) and returns a human-comparable performance.