InternLM2 Technical Report

TL;DR

InternLM2 is a new open-source large language model that surpasses previous models in multiple benchmarks, long-context understanding, and subjective evaluations through innovative training and alignment techniques.

Contribution

It introduces InternLM2 with advanced pre-training, long-context modeling, and a novel COOL RLHF alignment method, providing a comprehensive open-source LLM with detailed training insights.

Findings

Outperforms predecessors on 30 benchmarks and 6 evaluation dimensions.

Effectively models long contexts up to 32k tokens.

Demonstrates strong performance on the Needle-in-a-Haystack test.

Abstract

The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI). However, replicating such advancements in open-source models has been challenging. This paper introduces InternLM2, an open-source LLM that outperforms its predecessors in comprehensive evaluations across 6 dimensions and 30 benchmarks, long-context modeling, and open-ended subjective evaluations through innovative pre-training and optimization techniques. The pre-training process of InternLM2 is meticulously detailed, highlighting the preparation of diverse data types including text, code, and long-context data. InternLM2 efficiently captures long-term dependencies, initially trained on 4k tokens before advancing to 32k tokens in pre-training and fine-tuning stages, exhibiting remarkable performance on the 200k ``Needle-in-a-Haystack" test. InternLM2 is further aligned using Supervised Fine-Tuning (SFT) and a novel Conditional Online Reinforcement Learning from Human Feedback (COOL RLHF) strategy that addresses conflicting human preferences and reward hacking. By releasing InternLM2 models in different training stages and model sizes, we provide the community with insights into the model's evolution.