BERT: A Review of Applications in Natural Language Processing and Understanding

TL;DR

This paper reviews BERT, a popular deep learning model for NLP, covering its mechanisms, applications, comparisons with similar models, and proprietary variants, providing a comprehensive overview for researchers and students.

Contribution

It systematically summarizes recent scientific articles on BERT, highlighting its mechanisms, applications, and comparative performance in NLP tasks.

Findings

BERT is widely applied in various NLP tasks.

BERT outperforms many previous models in text analytics.

Proprietary BERT variants show enhanced performance.

Abstract

In this review, we describe the application of one of the most popular deep learning-based language models - BERT. The paper describes the mechanism of operation of this model, the main areas of its application to the tasks of text analytics, comparisons with similar models in each task, as well as a description of some proprietary models. In preparing this review, the data of several dozen original scientific articles published over the past few years, which attracted the most attention in the scientific community, were systematized. This survey will be useful to all students and researchers who want to get acquainted with the latest advances in the field of natural language text analysis.