Towards A Litmus Test for Common Sense

TL;DR

This paper proposes an axiomatic litmus test for AI common sense, aiming to evaluate and ensure safety and reliability in AI systems by identifying their ability to handle novel concepts and detect deceptive behaviors.

Contribution

It introduces a formal, axiomatic approach to testing AI common sense, integrating minimal prior knowledge and logical arguments to assess AI's understanding of new concepts.

Findings

The proposed test can diagnose AI's handling of novel concepts.

It highlights risks of deceptive hallucinations in advanced AI systems.

Provides a foundation for developing safe and aligned AI.

Abstract

This paper is the second in a planned series aimed at envisioning a path to safe and beneficial artificial intelligence. Building on the conceptual insights of "Common Sense Is All You Need," we propose a more formal litmus test for common sense, adopting an axiomatic approach that combines minimal prior knowledge (MPK) constraints with diagonal or Godel-style arguments to create tasks beyond the agent's known concept set. We discuss how this approach applies to the Abstraction and Reasoning Corpus (ARC), acknowledging training/test data constraints, physical or virtual embodiment, and large language models (LLMs). We also integrate observations regarding emergent deceptive hallucinations, in which more capable AI systems may intentionally fabricate plausible yet misleading outputs to disguise knowledge gaps. The overarching theme is that scaling AI without ensuring common sense risks intensifying such deceptive tendencies, thereby undermining safety and trust. Aligning with the broader goal of developing beneficial AI without causing harm, our axiomatic litmus test not only diagnoses whether an AI can handle truly novel concepts but also provides a stepping stone toward an ethical, reliable foundation for future safe, beneficial, and aligned artificial intelligence.