A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Katikapalli Subramanyam Kalyan

TL;DR
This survey comprehensively reviews GPT-3 family large language models, including ChatGPT and GPT-4, covering their architectures, capabilities, performance across tasks, and future research directions.
Contribution
It provides a detailed overview of GLLMs, summarizing recent research progress and offering insights into their applications, robustness, and future challenges.
Findings
GLLMs excel in various NLP tasks without task-specific training.
They demonstrate strong performance across multiple languages and domains.
The survey highlights key future research directions for GLLMs.
Abstract
Large language models (LLMs) are a special class of pretrained language models obtained by scaling model size, pretraining corpus and computation. LLMs, because of their large size and pretraining on large volumes of text data, exhibit special abilities which allow them to achieve remarkable performances without any task-specific training in many of the natural language processing tasks. The era of LLMs started with OpenAI GPT-3 model, and the popularity of LLMs is increasing exponentially after the introduction of models like ChatGPT and GPT4. We refer to GPT-3 and its successor OpenAI models, including ChatGPT and GPT4, as GPT-3 family large language models (GLLMs). With the ever-rising popularity of GLLMs, especially in the research community, there is a strong need for a comprehensive survey which summarizes the recent research progress in multiple dimensions and can guide the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Machine Learning in Healthcare
MethodsAttention Is All You Need · Cosine Annealing · Layer Normalization · Refunds@Expedia|||How do I get a full refund from Expedia? · Linear Warmup With Cosine Annealing · Linear Layer · Softmax · Dense Connections · {Dispute@FaQ-s}How to file a dispute with Expedia? · Residual Connection
