OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

TL;DR

OpenFlamingo is an open-source framework that replicates large autoregressive vision-language models, achieving comparable performance to DeepMind's Flamingo on multiple datasets, and provides accessible models and code for the community.

Contribution

It introduces an open-source family of large vision-language models that closely replicate Flamingo's performance, facilitating broader research and development.

Findings

Achieves 80-89% of Flamingo's performance on seven datasets.

Provides open-source models and training code.

Demonstrates effective replication of proprietary models.

Abstract

We introduce OpenFlamingo, a family of autoregressive vision-language models ranging from 3B to 9B parameters. OpenFlamingo is an ongoing effort to produce an open-source replication of DeepMind's Flamingo models. On seven vision-language datasets, OpenFlamingo models average between 80 - 89% of corresponding Flamingo performance. This technical report describes our models, training data, hyperparameters, and evaluation suite. We share our models and code at https://github.com/mlfoundations/open_flamingo.