Evaluating Nova 2.0 Lite model under Amazon's Frontier Model Safety Framework

TL;DR

This paper evaluates Nova 2.0 Lite's safety and capabilities under Amazon’s Frontier Model Safety Framework, using automated benchmarks, expert red-teaming, and uplift studies across high-risk domains.

Contribution

It provides a comprehensive safety evaluation of Nova 2.0 Lite, a highly capable reasoning model processing text, images, and videos with a 1M token context.

Findings

Nova 2.0 Lite meets safety thresholds in high-risk domains

Automated benchmarks and red-teaming identify potential risks

The evaluation methodology will be extended for future models

Abstract

Amazon published its Frontier Model Safety Framework (FMSF) as part of the Paris AI summit, following which we presented a report on Amazon's Premier model. In this report, we present an evaluation of Nova 2.0 Lite. Nova 2.0 Lite was made generally available from amongst the Nova 2.0 series and is one of its most capable reasoning models. The model processes text, images, and video with a context length of up to 1M tokens, enabling analysis of large codebases, documents, and videos in a single prompt. We present a comprehensive evaluation of Nova 2.0 Lite's critical risk profile under the FMSF. Evaluations target three high-risk domains-Chemical, Biological, Radiological and Nuclear (CBRN), Offensive Cyber Operations, and Automated AI R&D-and combine automated benchmarks, expert red-teaming, and uplift studies to determine whether the model exceeds release thresholds. We summarize our methodology and report core findings. We will continue to enhance our safety evaluation and mitigation pipelines as new risks and capabilities associated with frontier models are identified.