Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA

TL;DR

This paper introduces Moxin, an open-source large language model with variants for vision-language and Chinese tasks, emphasizing transparency and community collaboration, and demonstrates their superior performance through experiments.

Contribution

The paper presents Moxin, a fully open-source LLM with variants for vision-language and Chinese tasks, developed with transparency and open data, advancing open-source AI capabilities.

Findings

Moxin variants outperform existing models in various benchmarks.

Models are trained using open-source frameworks and data.

All models, data, and code are publicly released.

Abstract

Recently, Large Language Models (LLMs) have undergone a significant transformation, marked by a rapid rise in both their popularity and capabilities. Leading this evolution are proprietary LLMs like GPT-4 and GPT-o1, which have captured widespread attention in the AI community due to their remarkable performance and versatility. Simultaneously, open-source LLMs, such as LLaMA and Mistral, have made great contributions to the ever-increasing popularity of LLMs due to the ease to customize and deploy the models across diverse applications. Moxin 7B is introduced as a fully open-source LLM developed in accordance with the Model Openness Framework, which moves beyond the simple sharing of model weights to embrace complete transparency in training, datasets, and implementation detail, thus fostering a more inclusive and collaborative research environment that can sustain a healthy open-source ecosystem. To further equip Moxin with various capabilities in different tasks, we develop three variants based on Moxin, including Moxin-VLM, Moxin-VLA, and Moxin-Chinese, which target the vision-language, vision-language-action, and Chinese capabilities, respectively. Experiments show that our models achieve superior performance in various evaluations. We adopt open-source framework and open data for the training. We release our models, along with the available data and code to derive these models.