A Comparative Analysis of Ethical and Safety Gaps in LLMs using Relative Danger Coefficient

TL;DR

This paper introduces the Relative Danger Coefficient metric to compare ethical and safety risks across various large language models, emphasizing the importance of human oversight in high-stakes scenarios.

Contribution

It presents a novel metric, the Relative Danger Coefficient, for assessing harm in LLMs and provides a comparative analysis of multiple models' ethical performance.

Findings

DeepSeek-V3(R1) shows improved reasoning capabilities.

GPT variants exhibit varying safety profiles.

The RDC metric effectively quantifies potential harm.

Abstract

Artificial Intelligence (AI) and Large Language Models (LLMs) have rapidly evolved in recent years, showcasing remarkable capabilities in natural language understanding and generation. However, these advancements also raise critical ethical questions regarding safety, potential misuse, discrimination and overall societal impact. This article provides a comparative analysis of the ethical performance of various AI models, including the brand new DeepSeek-V3(R1 with reasoning and without), various GPT variants (4o, 3.5 Turbo, 4 Turbo, o1/o3 mini) and Gemini (1.5 flash, 2.0 flash and 2.0 flash exp) and highlights the need for robust human oversight, especially in situations with high stakes. Furthermore, we present a new metric for calculating harm in LLMs called Relative Danger Coefficient (RDC).