FONTNET: On-Device Font Understanding and Prediction Pipeline

TL;DR

This paper introduces FONTNET, an on-device system with novel CNN architecture and algorithms for font detection and prediction, enabling real-time, privacy-preserving font understanding in images with high efficiency.

Contribution

The paper presents a novel CNN architecture for font style identification, a new algorithm for font similarity prediction, and an optimized on-device deployment for real-time applications.

Findings

Achieved 30ms inference time on device

Developed a 4.5MB model for font tasks

Enabled privacy-preserving font analysis in real time

Abstract

Fonts are one of the most basic and core design concepts. Numerous use cases can benefit from an in depth understanding of Fonts such as Text Customization which can change text in an image while maintaining the Font attributes like style, color, size. Currently, Text recognition solutions can group recognized text based on line breaks or paragraph breaks, if the Font attributes are known multiple text blocks can be combined based on context in a meaningful manner. In this paper, we propose two engines: Font Detection Engine, which identifies the font style, color and size attributes of text in an image and a Font Prediction Engine, which predicts similar fonts for a query font. Major contributions of this paper are three-fold: First, we developed a novel CNN architecture for identifying font style of text in images. Second, we designed a novel algorithm for predicting similar fonts for a given query font. Third, we have optimized and deployed the entire engine On-Device which ensures privacy and improves latency in real time applications such as instant messaging. We achieve a worst case On-Device inference time of 30ms and a model size of 4.5MB for both the engines.