BrailleLLM: Braille Instruction Tuning with Large Language Models for Braille Domain Tasks

TL;DR

BrailleLLM introduces a novel fine-tuning approach using large language models and new datasets to improve Braille translation and understanding, addressing data scarcity and ambiguity issues in Braille processing.

Contribution

The paper presents BrailleLLM, a new instruction tuning method with datasets and augmentation techniques specifically designed for Braille domain tasks.

Findings

BKFT significantly outperforms traditional fine-tuning in Braille translation.

Constructed multilingual Braille datasets support diverse research.

Methodology establishes a foundation for low-resource Braille research.

Abstract

Braille plays a vital role in education and information accessibility for visually impaired individuals. However, Braille information processing faces challenges such as data scarcity and ambiguities in mixed-text contexts. We construct English and Chinese Braille Mixed Datasets (EBMD/CBMD) with mathematical formulas to support diverse Braille domain research, and propose a syntax tree-based augmentation method tailored for Braille data. To address the underperformance of traditional fine-tuning methods in Braille-related tasks, we investigate Braille Knowledge-Based Fine-Tuning (BKFT), which reduces the learning difficulty of Braille contextual features. BrailleLLM employs BKFT via instruction tuning to achieve unified Braille translation, formula-to-Braille conversion, and mixed-text translation. Experiments demonstrate that BKFT achieves significant performance improvements over conventional fine-tuning in Braille translation scenarios. Our open-sourced datasets and methodologies establish a foundation for low-resource multilingual Braille research.