Aligning Human and Machine Attention for Enhanced Supervised Learning

TL;DR

This paper introduces HuMAL, a novel method that incorporates human attention data into machine learning models, significantly improving their performance especially with limited or imbalanced data, by aligning machine attention with human attention mechanisms.

Contribution

It presents a new approach called HuMAL that effectively integrates human attention annotations into ML models, enhancing their performance on NLP tasks.

Findings

HuMAL improves transformer model performance on sentiment and personality classification tasks.

The approach is especially effective with sparse or imbalanced labeled data.

Aligning machine attention with human attention enhances learning outcomes.

Abstract

Attention, or prioritization of certain information items over others, is a critical element of any learning process, for both humans and machines. Given that humans continue to outperform machines in certain learning tasks, it seems plausible that machine performance could be enriched by aligning machine attention with human attention mechanisms -- yet research on this topic is sparse and has achieved only limited success. This paper proposes a new approach to address this gap, called Human-Machine Attention Learning (HuMAL). This approach involves reliance on data annotated by humans to reflect their self-perceived attention during specific tasks. We evaluate several alternative strategies for integrating such human attention data into machine learning (ML) algorithms, using a sentiment analysis task (review data from Yelp) and a personality-type classification task (data from myPersonality). The best-performing HuMAL strategy significantly enhances the task performance of fine-tuned transformer models (BERT, as well as GPT-2 and XLNET), and the benefit is particularly pronounced under challenging conditions of imbalanced or sparse labeled data. This research contributes to a deeper understanding of strategies for integrating human attention into ML models and highlights the potential of leveraging human cognition to augment ML in real-world applications.