A Hybrid Framework for Matching Printing Design Files to Product Photos

TL;DR

This paper introduces a real-time hybrid image matching framework combining hand-crafted and deep features, specifically designed for matching printing design files to product photos despite various image distortions.

Contribution

It presents a novel hybrid framework that leverages deep learning for improved accuracy while maintaining real-time performance for printing design to photo matching.

Findings

Deep features outperform hand-crafted features in matching accuracy.

The framework achieves real-time performance on standard desktop hardware.

Benchmark results demonstrate robustness against common image distortions.

Abstract

We propose a real-time image matching framework, which is hybrid in the sense that it uses both hand-crafted features and deep features obtained from a well-tuned deep convolutional network. The matching problem, which we concentrate on, is specific to a certain application, that is, printing design to product photo matching. Printing designs are any kind of template image files, created using a design tool, thus are perfect image signals. However, photographs of a printed product suffer many unwanted effects, such as uncontrolled shooting angle, uncontrolled illumination, occlusions, printing deficiencies in color, camera noise, optic blur, et cetera. For this purpose, we create an image set that includes printing design and corresponding product photo pairs with collaboration of an actual printing facility. Using this image set, we benchmark various hand-crafted and deep features for matching performance and propose a framework in which deep learning is utilized with highest contribution, but without disabling real-time operation using an ordinary desktop computer.