OracleFusion: Assisting the Decipherment of Oracle Bone Script with Structurally Constrained Semantic Typography

TL;DR

OracleFusion is a novel two-stage framework that combines multimodal analysis and structural constraints to improve the decipherment and visualization of Oracle Bone Script characters, aiding experts in interpretation.

Contribution

This paper introduces OracleFusion, a new approach integrating multimodal large language models and structural vector fusion to enhance OBS decipherment and glyph visualization.

Findings

Outperforms baseline models in semantics and visual quality

Enhances readability and aesthetic appeal of glyph representations

Provides insights on unseen oracle characters

Abstract

As one of the earliest ancient languages, Oracle Bone Script (OBS) encapsulates the cultural records and intellectual expressions of ancient civilizations. Despite the discovery of approximately 4,500 OBS characters, only about 1,600 have been deciphered. The remaining undeciphered ones, with their complex structure and abstract imagery, pose significant challenges for interpretation. To address these challenges, this paper proposes a novel two-stage semantic typography framework, named OracleFusion. In the first stage, this approach leverages the Multimodal Large Language Model (MLLM) with enhanced Spatial Awareness Reasoning (SAR) to analyze the glyph structure of the OBS character and perform visual localization of key components. In the second stage, we introduce Oracle Structural Vector Fusion (OSVF), incorporating glyph structure constraints and glyph maintenance constraints to ensure the accurate generation of semantically enriched vector fonts. This approach preserves the objective integrity of the glyph structure, offering visually enhanced representations that assist experts in deciphering OBS. Extensive qualitative and quantitative experiments demonstrate that OracleFusion outperforms state-of-the-art baseline models in terms of semantics, visual appeal, and glyph maintenance, significantly enhancing both readability and aesthetic quality. Furthermore, OracleFusion provides expert-like insights on unseen oracle characters, making it a valuable tool for advancing the decipherment of OBS.