A Benchmarking Framework for AI models in Automotive Aerodynamics

TL;DR

This paper presents an open-source benchmarking framework for AI models in automotive aerodynamics, enabling systematic evaluation of accuracy, performance, and generalization to accelerate research and development.

Contribution

It introduces a standardized, extensible benchmarking framework within NVIDIA's PhysicsNeMo-CFD for assessing AI models in automotive aerodynamics.

Findings

Evaluated three AI models on the DrivAerML dataset.

Provided guidelines for incorporating additional models and datasets.

Enhanced transparency and comparability in AI model assessment.

Abstract

In this paper, we introduce a benchmarking framework within the open-source NVIDIA PhysicsNeMo-CFD framework designed to systematically assess the accuracy, performance, scalability, and generalization capabilities of AI models for automotive aerodynamics predictions. The open extensible framework enables incorporation of a diverse set of metrics relevant to the Computer-Aided Engineering (CAE) community. By providing a standardized methodology for comparing AI models, the framework enhances transparency and consistency in performance assessment, with the overarching goal of improving the understanding and development of these models to accelerate research and innovation in the field. To demonstrate its utility, the framework includes evaluation of both surface and volumetric flow field predictions on three AI models: DoMINO, X-MeshGraphNet, and FIGConvNet using the DrivAerML dataset. It also includes guidelines for integrating additional models and datasets, making it extensible for physically consistent metrics. This benchmarking study aims to enable researchers and industry professionals in selecting, refining, and advancing AI-driven aerodynamic modeling approaches, ultimately fostering the development of more efficient, accurate, and interpretable solutions in automotive aerodynamics