The Resale Price Prediction of Secondhand Jewelry Items Using a Multi-modal Deep Model with Iterative Co-Attention

TL;DR

This paper introduces a multimodal deep learning model with iterative co-attention to predict resale prices of secondhand jewelry, eliminating the need for expert knowledge and leveraging images and attributes for fine-grained assessment.

Contribution

The paper presents a novel multimodal deep neural network with iterative co-attention for autonomous resale price prediction of jewelry, improving accuracy over simple fusion methods.

Findings

Effective resale price prediction achieved on a large jewelry dataset.

Iterative co-attention enhances the model's focus on relevant features.

Model architecture applicable to other fashion items with visual and attribute data.

Abstract

The resale price assessment of secondhand jewelry items relies heavily on the individual knowledge and skill of domain experts. In this paper, we propose a methodology for reconstructing an AI system that autonomously assesses the resale prices of secondhand jewelry items without the need for professional knowledge. As shown in recent studies on fashion items, multimodal approaches combining specifications and visual information of items have succeeded in obtaining fine-grained representations of fashion items, although they generally apply simple vector operations through a multimodal fusion. We similarly build a multimodal model using images and attributes of the product and further employ state-of-the-art multimodal deep neural networks applied in computer vision to achieve a practical performance level. In addition, we model the pricing procedure of an expert using iterative co-attention networks in which the appearance and attributes of the product are carefully and iteratively observed. Herein, we demonstrate the effectiveness of our model using a large dataset of secondhand no brand jewelry items received from a collaborating fashion retailer, and show that the iterative co-attention process operates effectively in the context of resale price prediction. Our model architecture is widely applicable to other fashion items where appearance and specifications are important aspects.