NVIDIA Nemotron Nano V2 VL

TL;DR

Nemotron Nano V2 VL is a new vision-language model optimized for real-world document and video understanding, offering significant improvements over previous models through architecture, dataset, and training enhancements.

Contribution

Introduces Nemotron Nano V2 VL with advanced architecture, token reduction, and training methods for improved long document and video comprehension.

Findings

Enhanced inference throughput in long scenarios

Significant performance improvements over previous models

Open release of datasets, recipes, and training code

Abstract

We introduce Nemotron Nano V2 VL, the latest model of the Nemotron vision-language series designed for strong real-world document understanding, long video comprehension, and reasoning tasks. Nemotron Nano V2 VL delivers significant improvements over our previous model, Llama-3.1-Nemotron-Nano-VL-8B, across all vision and text domains through major enhancements in model architecture, datasets, and training recipes. Nemotron Nano V2 VL builds on Nemotron Nano V2, a hybrid Mamba-Transformer LLM, and innovative token reduction techniques to achieve higher inference throughput in long document and video scenarios. We are releasing model checkpoints in BF16, FP8, and FP4 formats and sharing large parts of our datasets, recipes and training code.