HiCoRe: Visual Hierarchical Context-Reasoning

TL;DR

This paper introduces HiCoRe, a hierarchical context reasoning framework that enhances image understanding by modeling hierarchical interactions with graph convolutional networks, significantly improving performance over traditional CNNs.

Contribution

It proposes a novel hierarchical reasoning framework that integrates graph-based models with CNNs, capturing multi-level interactions beyond fine-grained labels.

Findings

Achieves three times greater accuracy on standard datasets.

Effectively models hierarchical interactions at multiple abstraction levels.

Demonstrates robustness across different granularity scenarios.

Abstract

Reasoning about images/objects and their hierarchical interactions is a key concept for the next generation of computer vision approaches. Here we present a new framework to deal with it through a visual hierarchical context-based reasoning. Current reasoning methods use the fine-grained labels from images' objects and their interactions to predict labels to new objects. Our framework modifies this current information flow. It goes beyond and is independent of the fine-grained labels from the objects to define the image context. It takes into account the hierarchical interactions between different abstraction levels (i.e. taxonomy) of information in the images and their bounding-boxes. Besides these connections, it considers their intrinsic characteristics. To do so, we build and apply graphs to graph convolution networks with convolutional neural networks. We show a strong effectiveness over widely used convolutional neural networks, reaching a gain 3 times greater on well-known image datasets. We evaluate the capability and the behavior of our framework under different scenarios, considering distinct (superclass, subclass and hierarchical) granularity levels. We also explore attention mechanisms through graph attention networks and pre-processing methods considering dimensionality expansion and/or reduction of the features' representations. Further analyses are performed comparing supervised and semi-supervised approaches.