Understanding and Estimating Domain Complexity Across Domains

TL;DR

This paper introduces a comprehensive framework to estimate and analyze domain complexity in AI, distinguishing intrinsic and extrinsic factors to better predict AI performance in diverse, real-world environments.

Contribution

It presents a novel, general framework for quantifying domain complexity by analyzing intrinsic and extrinsic factors, aiding AI performance prediction across various environments.

Findings

Framework effectively differentiates intrinsic and extrinsic complexities.

Enables quantitative prediction of AI difficulty during environment transitions.

Helps navigate large search spaces in open-world domains.

Abstract

Artificial Intelligence (AI) systems, trained in controlled environments, often struggle in real-world complexities. We propose a general framework for estimating domain complexity across diverse environments, like open-world learning and real-world applications. This framework distinguishes between intrinsic complexity (inherent to the domain) and extrinsic complexity (dependent on the AI agent). By analyzing dimensionality, sparsity, and diversity within these categories, we offer a comprehensive view of domain challenges. This approach enables quantitative predictions of AI difficulty during environment transitions, avoids bias in novel situations, and helps navigate the vast search spaces of open-world domains.