AI, orthogonality and the M\"uller-Cannon instrumental vs general intelligence distinction

TL;DR

This paper critically examines the ongoing debate about AI's existential risks, reaffirming the validity of the concern despite recent claims of a flaw by Muller and Cannon.

Contribution

It provides a careful conceptual analysis defending the standard argument for AI existential risk against recent critiques.

Findings

The standard AI risk argument remains valid.

Recent critique by Muller and Cannon is addressed and rebutted.

The distinction between instrumental and general intelligence is clarified.

Abstract

The by now standard argument put forth by Yudkowsky, Bostrom and others for why the possibility of a carelessly handled AI breakthrough poses an existential threat to humanity is shown through careful conceptual analysis to be very much alive and kicking, despite the suggestion in a recent paper by M\"uller and Cannon that the argument contains a flaw.