AesCrop: Aesthetic-driven Cropping Guided by Composition

TL;DR

AesCrop is a novel image cropping method that uses a composition-aware transformer model to generate aesthetically pleasing crops by explicitly encoding photographic composition cues, outperforming existing methods.

Contribution

The paper introduces AesCrop, a hybrid image cropping model that incorporates a novel composition attention bias to explicitly encode photographic composition cues.

Findings

AesCrop achieves superior quantitative metrics compared to state-of-the-art methods.

AesCrop produces more aesthetically pleasing crops qualitatively.

The model effectively focuses on salient compositional regions in images.

Abstract

Aesthetic-driven image cropping is crucial for applications like view recommendation and thumbnail generation, where visual appeal significantly impacts user engagement. A key factor in visual appeal is composition--the deliberate arrangement of elements within an image. Some methods have successfully incorporated compositional knowledge through evaluation-based and regression-based paradigms. However, evaluation-based methods lack globality while regression-based methods lack diversity. Recently, hybrid approaches that integrate both paradigms have emerged, bridging the gap between these two to achieve better diversity and globality. Notably, existing hybrid methods do not incorporate photographic composition guidance, a key attribute that defines photographic aesthetics. In this work, we introduce AesCrop, a composition-aware hybrid image-cropping model that integrates a VMamba image encoder, augmented with a novel Mamba Composition Attention Bias (MCAB) and a transformer decoder to perform end-to-end rank-based image cropping, generating multiple crops along with the corresponding quality scores. By explicitly encoding compositional cues into the attention mechanism, MCAB directs AesCrop to focus on the most compositionally salient regions. Extensive experiments demonstrate that AesCrop outperforms current state-of-the-art methods, delivering superior quantitative metrics and qualitatively more pleasing crops.