Free Lunch for Efficient Textual Commonsense Integration in Language Models

TL;DR

This paper introduces an efficient batching method for integrating textual commonsense knowledge into language models, reducing computational costs without sacrificing performance, especially beneficial for large datasets and memory-constrained devices.

Contribution

It proposes a spectral clustering-based batching approach that reuses commonsense descriptions across samples, optimizing efficiency without altering the language model.

Findings

Significant reduction in computational cost observed.

Efficiency gains are larger on bigger datasets.

Performance is preserved despite batching optimization.

Abstract

Recent years have witnessed the emergence of textual commonsense knowledge bases, aimed at providing more nuanced and context-rich knowledge. The integration of external commonsense into language models has been shown to be a key enabler in advancing the state-of-the-art for a wide range of NLP tasks. However, incorporating textual commonsense descriptions is computationally expensive, as compared to encoding conventional symbolic knowledge. In this paper, we propose a method to improve its efficiency without modifying the model. We group training samples with similar commonsense descriptions into a single batch, thus reusing the encoded description across multiple samples. One key observation is that the upper bound of batch partitioning can be reduced to the classic {\it graph k-cut problem}. Consequently, we propose a spectral clustering-based algorithm to solve this problem. Extensive experiments illustrate that the proposed batch partitioning approach effectively reduces the computational cost while preserving performance. The efficiency improvement is more pronounced on larger datasets and on devices with more memory capacity, attesting to its practical utility for large-scale applications.