PropNet: a White-Box and Human-Like Network for Sentence Representation

TL;DR

PropNet is a novel white-box, human-like sentence representation model inspired by cognitive science, aiming to improve interpretability and understanding of human cognition in NLP tasks, despite current performance gaps.

Contribution

It introduces a hierarchical, interpretable network for sentence embedding based on propositions, addressing black-box issues of existing models.

Findings

PropNet has a significant gap in STS performance compared to SOTA models.

Case studies demonstrate PropNet's potential for understanding human cognitive processes.

PropNet enables analysis of the cognitive basis of sentence similarity.

Abstract

Transformer-based embedding methods have dominated the field of sentence representation in recent years. Although they have achieved remarkable performance on NLP missions, such as semantic textual similarity (STS) tasks, their black-box nature and large-data-driven training style have raised concerns, including issues related to bias, trust, and safety. Many efforts have been made to improve the interpretability of embedding models, but these problems have not been fundamentally resolved. To achieve inherent interpretability, we propose a purely white-box and human-like sentence representation network, PropNet. Inspired by findings from cognitive science, PropNet constructs a hierarchical network based on the propositions contained in a sentence. While experiments indicate that PropNet has a significant gap compared to state-of-the-art (SOTA) embedding models in STS tasks, case studies reveal substantial room for improvement. Additionally, PropNet enables us to analyze and understand the human cognitive processes underlying STS benchmarks.