Evolution of Natural Language Processing Technology: Not Just Language Processing Towards General Purpose AI

TL;DR

This paper discusses the evolution of NLP technology from language processing to general-purpose AI, highlighting deep learning's role in achieving unprecedented results and practical applications.

Contribution

It provides a technological overview of recent NLP advancements, emphasizing how deep learning enables models to learn complex tasks without explicit programming.

Findings

Deep learning has enabled NLP models to perform arithmetic and image explanation tasks.

Large textual data training leads to models that embody the 'practice makes perfect' principle.

The paper summarizes recent NLP developments leading to large language models.

Abstract

Since the invention of computers, communication through natural language (actual human language) has been a dream technology. However, natural language is extremely difficult to mathematically formulate, making it difficult to realize as an algorithm without considering programming. While there have been numerous technological developments, one cannot say that any results allowing free utilization have been achieved thus far. In the case of language learning in humans, for instance when learning one's mother tongue or foreign language, one must admit that this process is similar to the adage "practice makes perfect" in principle, even though the learning method is significant up to a point. Deep learning has played a central role in contemporary AI technology in recent years. When applied to natural language processing (NLP), this produced unprecedented results. Achievements exceeding the initial predictions have been reported from the results of learning vast amounts of textual data using deep learning. For instance, four arithmetic operations could be performed without explicit learning, thereby enabling the explanation of complex images and the generation of images from corresponding explanatory texts. It is an accurate example of the learner embodying the concept of "practice makes perfect" by using vast amounts of textual data. This report provides a technological explanation of how cutting-edge NLP has made it possible to realize the "practice makes perfect" principle. Additionally, examples of how this can be applied to business are provided. We reported in June 2022 in Japanese on the NLP movement from late 2021 to early 2022. We would like to summarize this as a memorandum since this is just the initial movement leading to the current large language models (LLMs).