ChartFormer: A Large Vision Language Model for Converting Chart Images into Tactile Accessible SVGs

TL;DR

ChartFormer is a novel model that converts raster chart images into tactile-accessible SVGs, enhancing accessibility for visually impaired users by enabling easier creation of tactile graphics.

Contribution

The paper introduces ChartFormer, a new model trained on a synthetic dataset to generate tactile SVGs from chart images, advancing accessible data visualization.

Findings

Effective conversion of raster charts to tactile SVGs

Positive user feedback from tactile display pilot study

Open-source implementation available for public use

Abstract

Visualizations, such as charts, are crucial for interpreting complex data. However, they are often provided as raster images, which are not compatible with assistive technologies for people with blindness and visual impairments, such as embossed papers or tactile displays. At the same time, creating accessible vector graphics requires a skilled sighted person and is time-intensive. In this work, we leverage advancements in the field of chart analysis to generate tactile charts in an end-to-end manner. Our three key contributions are as follows: (1) introducing the ChartFormer model trained to convert raster chart images into tactile-accessible SVGs, (2) training this model on the Chart2Tactile dataset, a synthetic chart dataset we created following accessibility standards, and (3) evaluating the effectiveness of our SVGs through a pilot user study with an refreshable two-dimensional tactile display. Our work is publicly available at https://github.com/nsothman/ChartFormer .