Enhancing Augmentative and Alternative Communication with Card Prediction and Colourful Semantics

TL;DR

This paper enhances AAC systems by integrating Colourful Semantics with a transformer-based model for Brazilian Portuguese, significantly improving communication card prediction accuracy and contextual relevance.

Contribution

The paper introduces BERTptCS, a novel model combining Colourful Semantics with transformer architecture for improved AAC communication in Portuguese.

Findings

BERTptCS outperforms baseline models in accuracy metrics.

Integration of CS improves contextual understanding.

Significant gains in top-k accuracy and MRR.

Abstract

This paper presents an approach to enhancing Augmentative and Alternative Communication (AAC) systems by integrating Colourful Semantics (CS) with transformer-based language models specifically tailored for Brazilian Portuguese. We introduce an adapted BERT model, BERTptCS, which incorporates the CS framework for improved prediction of communication cards. The primary aim is to enhance the accuracy and contextual relevance of communication card predictions, which are essential in AAC systems for individuals with complex communication needs (CCN). We compared BERTptCS with a baseline model, BERTptAAC, which lacks CS integration. Our results demonstrate that BERTptCS significantly outperforms BERTptAAC in various metrics, including top-k accuracy, Mean Reciprocal Rank (MRR), and Entropy@K. Integrating CS into the language model improves prediction accuracy and offers a more intuitive and contextual understanding of user inputs, facilitating more effective communication.