Attention based Writer Independent Handwriting Verification

TL;DR

This paper introduces an attention-based neural network for writer-independent handwriting verification that provides interpretable decision explanations and achieves high accuracy on a cursive dataset.

Contribution

It integrates cross-attention and soft-attention mechanisms to enhance interpretability and performance in handwriting verification tasks.

Findings

Achieved 86% precision on CEDAR cursive dataset

Generated meaningful attention-based explanations

Improved focus on relevant input areas

Abstract

The task of writer verification is to provide a likelihood score for whether the queried and known handwritten image samples belong to the same writer or not. Such a task calls for the neural network to make it's outcome interpretable, i.e. provide a view into the network's decision making process. We implement and integrate cross-attention and soft-attention mechanisms to capture the highly correlated and salient points in feature space of 2D inputs. The attention maps serve as an explanation premise for the network's output likelihood score. The attention mechanism also allows the network to focus more on relevant areas of the input, thus improving the classification performance. Our proposed approach achieves a precision of 86\% for detecting intra-writer cases in CEDAR cursive "AND" dataset. Furthermore, we generate meaningful explanations for the provided decision by extracting attention maps from multiple levels of the network.