Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research

TL;DR

Transformer language models (TLMs) have the potential to significantly advance information systems research by improving text analysis, enabling new research topics, and supporting multilingual applications, thus broadening AI's impact on business and society.

Contribution

The paper reviews recent progress in TLMs and proposes how they can enhance and expand IS research, highlighting their advantages over traditional text mining methods.

Findings

TLMs outperform existing text mining techniques in various tasks.

Multilingual models enable higher quality analysis across languages.

Advanced models facilitate development of custom, powerful systems.

Abstract

AI is widely thought to be poised to transform business, yet current perceptions of the scope of this transformation may be myopic. Recent progress in natural language processing involving transformer language models (TLMs) offers a potential avenue for AI-driven business and societal transformation that is beyond the scope of what most currently foresee. We review this recent progress as well as recent literature utilizing text mining in top IS journals to develop an outline for how future IS research can benefit from these new techniques. Our review of existing IS literature reveals that suboptimal text mining techniques are prevalent and that the more advanced TLMs could be applied to enhance and increase IS research involving text data, and to enable new IS research topics, thus creating more value for the research community. This is possible because these techniques make it easier to develop very powerful custom systems and their performance is superior to existing methods for a wide range of tasks and applications. Further, multilingual language models make possible higher quality text analytics for research in multiple languages. We also identify new avenues for IS research, like language user interfaces, that may offer even greater potential for future IS research.