JIANG: Chinese Open Foundation Language Model

TL;DR

JIANG is a Chinese-specific large language model trained on a substantial Chinese corpus, optimized for better performance in Chinese language tasks, addressing limitations of existing models primarily trained on English data.

Contribution

We introduce JIANG, a Chinese-focused language model trained on a large Chinese corpus with optimized structure, enhancing Chinese language understanding and expression.

Findings

Demonstrates excellent performance in Chinese language tasks

Outperforms existing models in Chinese benchmarks

Shows significant improvement over English-trained models in Chinese tasks

Abstract

With the advancements in large language model technology, it has showcased capabilities that come close to those of human beings across various tasks. This achievement has garnered significant interest from companies and scientific research institutions, leading to substantial investments in the research and development of these models. While numerous large models have emerged during this period, the majority of them have been trained primarily on English data. Although they exhibit decent performance in other languages, such as Chinese, their potential remains limited due to factors like vocabulary design and training corpus. Consequently, their ability to fully express their capabilities in Chinese falls short. To address this issue, we introduce the model named JIANG (Chinese pinyin of ginger) specifically designed for the Chinese language. We have gathered a substantial amount of Chinese corpus to train the model and have also optimized its structure. The extensive experimental results demonstrate the excellent performance of our model.