RAISE: A Unified Framework for Responsible AI Scoring and Evaluation

TL;DR

RAISE introduces a comprehensive framework for evaluating AI models across multiple responsibility dimensions, providing a holistic score that aids in responsible AI deployment in high-stakes domains.

Contribution

The paper presents RAISE, a novel unified framework that quantifies and aggregates explainability, fairness, robustness, and sustainability into a single Responsibility Score.

Findings

MLP shows strong sustainability and robustness.

Transformer excels in explainability and fairness but is environmentally costly.

Tabular ResNet offers a balanced profile across criteria.

Abstract

As AI systems enter high-stakes domains, evaluation must extend beyond predictive accuracy to include explainability, fairness, robustness, and sustainability. We introduce RAISE (Responsible AI Scoring and Evaluation), a unified framework that quantifies model performance across these four dimensions and aggregates them into a single, holistic Responsibility Score. We evaluated three deep learning models: a Multilayer Perceptron (MLP), a Tabular ResNet, and a Feature Tokenizer Transformer, on structured datasets from finance, healthcare, and socioeconomics. Our findings reveal critical trade-offs: the MLP demonstrated strong sustainability and robustness, the Transformer excelled in explainability and fairness at a very high environmental cost, and the Tabular ResNet offered a balanced profile. These results underscore that no single model dominates across all responsibility criteria, highlighting the necessity of multi-dimensional evaluation for responsible model selection. Our implementation is available at: https://github.com/raise-framework/raise.