Learning Multi-level Deep Representations for Image Emotion Classification

TL;DR

This paper introduces MldrNet, a deep network that combines multi-level features like semantics, aesthetics, and low-level visual cues to improve image emotion classification accuracy across various image types.

Contribution

The paper presents a novel deep network that integrates multi-level features for more effective image emotion classification, outperforming existing methods.

Findings

Outperforms state-of-the-art methods by at least 6% accuracy

Effective across diverse image types including paintings and web images

Combines semantic, aesthetic, and low-level features for improved classification

Abstract

In this paper, we propose a new deep network that learns multi-level deep representations for image emotion classification (MldrNet). Image emotion can be recognized through image semantics, image aesthetics and low-level visual features from both global and local views. Existing image emotion classification works using hand-crafted features or deep features mainly focus on either low-level visual features or semantic-level image representations without taking all factors into consideration. The proposed MldrNet combines deep representations of different levels, i.e. image semantics, image aesthetics, and low-level visual features to effectively classify the emotion types of different kinds of images, such as abstract paintings and web images. Extensive experiments on both Internet images and abstract paintings demonstrate the proposed method outperforms the state-of-the-art methods using deep features or hand-crafted features. The proposed approach also outperforms the state-of-the-art methods with at least 6% performance improvement in terms of overall classification accuracy.