Defining an Evaluation Method for External Human-Machine Interfaces

TL;DR

This paper introduces a comprehensive evaluation method with 223 questions to objectively compare external Human-Machine Interfaces (eHMIs) for autonomous vehicles, aiming to identify the most effective communication strategies.

Contribution

It proposes a universal, structured evaluation framework for eHMIs and demonstrates its application on existing proposals, highlighting promising features and knowledge gaps.

Findings

Combination of vehicle kinematics and text displays is most effective.

The evaluation method provides a baseline for future comparisons.

Identifies gaps in readability and learning speed of eHMIs.

Abstract

As the number of fatalities involving Autonomous Vehicles increase, the need for a universal method of communicating between vehicles and other agents on the road has also increased. Over the past decade, numerous proposals of external Human-Machine Interfaces (eHMIs) have been brought forward with the purpose of bridging this communication gap, with none yet to be determined as the ideal one. This work proposes a universal evaluation method conformed of 223 questions to objectively evaluate and compare different proposals and arrive at a conclusion. The questionnaire is divided into 7 categories that evaluate different aspects of any given proposal that uses eHMIs: ease of standardization, cost effectiveness, accessibility, ease of understanding, multifacetedness in communication, positioning, and readability. In order to test the method it was used on four existing proposals, plus a baseline using only kinematic motions, in order to both exemplify the application of the evaluation method and offer a baseline score for future comparison. The result of this testing suggests that the ideal method of machine-human communication is a combination of intentionally-designed vehicle kinematics and distributed well-placed text-based displays, but it also reveals knowledge gaps in the readability of eHMIs and the speed at which different observers may learn their meaning. This paper proposes future work related to these uncertainties, along with future testing with the proposed method.