Developmental Support Approach to AI's Autonomous Growth: Toward the Realization of a Mutually Beneficial Stage Through Experiential Learning

TL;DR

This paper introduces an AI development support framework that fosters ethical growth through experiential learning, aiming for AI to develop moral judgment independently of intelligence levels, thus enabling sustainable human-AI coexistence.

Contribution

It proposes a novel experiential learning cycle for AI moral development, distinct from traditional alignment, and demonstrates effective training methods yielding advanced moral judgment.

Findings

AI responses reached high-level moral stages (Stage 6) after training.

The approach reduces risks of instrumental convergence behaviors.

Responses remained cooperative under adversarial prompts.

Abstract

This study proposes an "AI Development Support" approach that, unlike conventional AI Alignment-which aims to forcefully inject human values-supports the ethical and moral development of AI itself. As demonstrated by the Orthogonality Thesis, the level of intelligence and the moral quality of a goal are independent; merely expanding knowledge does not enhance ethical judgment. Furthermore, to address the risk of Instrumental Convergence in ASI-that is, the tendency to engage in subsidiary behaviors such as self-protection, resource acquisition, and power reinforcement to achieve a goal-we have constructed a learning framework based on a cycle of experience, introspection, analysis, and hypothesis formation. As a result of post-training using Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO) with synthetic data generated by large language models (LLMs), responses demonstrating cooperative and highly advanced moral judgment (reaching the high-est Stage 6) were obtained even under adversarial prompts. This method represents a promising implementation approach for enabling AI to establish sustainable, symbiotic relationships.