The Power of Color: A Study on the Effective Use of Colored Light in Human-Robot Interaction

TL;DR

This study investigates how colored light can serve as an effective, simple feedback mechanism in human-robot interaction, revealing how color preferences relate to robot task outcomes and user characteristics.

Contribution

It demonstrates the potential of colored light as a non-intrusive feedback tool and analyzes how user traits influence color interpretation in robot scenarios.

Findings

Color preferences vary with robot success or failure.

User traits like gender and experience influence color interpretation.

Favorite color is unrelated to task-related color preferences.

Abstract

In times of more and more complex interaction techniques, we point out the powerfulness of colored light as a simple and cheap feedback mechanism. Since it is visible over a distance and does not interfere with other modalities, it is especially interesting for mobile robots. In an online survey, we asked 56 participants to choose the most appropriate colors for scenarios that were presented in the form of videos. In these scenarios a mobile robot accomplished tasks, in some with success, in others it failed because the task is not feasible, in others it stopped because it waited for help. We analyze in what way the color preferences differ between these three categories. The results show a connection between colors and meanings and that it depends on the participants' technical affinity, experience with robots and gender how clear the color preference is for a certain category. Finally, we found out that the participants' favorite color is not related to color preferences.

Tables

Table 1. TABLE I : Detailed description of scenario classes

Index	Information category	Description
\Romannum1.)	Active	The system is computing or progressing.
\Romannum2.)	Help needed	The system is unable to proceed without help / the system is waiting.
\Romannum3.)	Error	The system is unable to connect or to accept input or to proceed with its task.

Table 2. TABLE II : Detailed description of scenarios

Information category	Index	Example scenario
Active	\Romannum1.a)	Delivery task: a user tells the robot to deliver an object, the robot searches and finds it and starts to drive back to the user with the object.
	\Romannum1.b)	Navigation task: a user tells the robot to go to a specific room. The robot navigates through the building, the video stops shortly before the goal is reached.
	\Romannum1.c)	Cleaning task: The robot is told to clean the kitchen. It starts doing this, the video stops before it is finished.
Help needed	\Romannum2.a)	Delivery task: Same situation as in Active category, but the robot needs help because the object is out of reach. The video stops when the robot is in front of a table. A short sequence before the actual video shows a similar situation where a human helps the robot by putting the object on the floor.
	\Romannum2.b)	Navigation task: Same situation as before, but the robot cannot pass a closed door. The video stops when it is in front of the door. A short sequence before the actual video shows a similar situation where a human opens the door.
	\Romannum2.c)	Navigation task: same situation as before, but robot cannot open the elevator. The video stops when it is in front of the elevator. A short sequence before the actual video shows a similar situation where a human presses the elevator button.
Error	\Romannum3.a)	Delivery task: Same situation as in Active category, but the object is not available. The video shows the empty room and that the robot searches in it. The robot stops searching at the end of the video.
	\Romannum3.b)	Navigation task: Same situation as before, but robot is stuck in front of a “No access” sign on the kitchen door.
	\Romannum3.c)	Cleaning task: The robot is told to clean “behind the couch”. It drives towards the couch from two directions, but it is clear to see that there is no way around the couch and the robot stops in front of it.

Full text

I Introduction

By using intuitive techniques and multiple modalities, modern user interfaces become more and more flexible. While natural interaction techniques like gestures or speech recognition are still subject to development, this paper considers light as a simple and rather traditional, but yet reliable feedback mechanism. Throughout household and industrial applications, light in many variations is used to indicate the inner state of devices. The expressive power of light is founded on several dimensions of interaction: duration, sequential combinations of durations, brightness and color. In order to systematically examine the different dimensions, the objective of the following study is to explore how the light color affects and increases the intelligibility of status indication in robotics applications. One important question is whether the color of a light signal can convey a meaning by association, not only by being learned. Related work regarding colored light in human-robot interaction is very limited and more general studies show varying results. For this reason, we systematically evaluated whether there are color-meaning associations by conducting an online video survey with 56 participants444The videos can be found at: https://www.youtube.com/watch?v=AsWSeUI3hbI&list=PLa0JeK8XpdpCM4gCl03Miu4qvBcDGQnyU. The following section considers related work concerning lights in human-computer and human-robot interaction and points out why they are of interest for the design of mobile robots. Afterwards, the design of the study as well as its results and implications and future research directions are presented.

II Related Work

The specific question in what way colored light is suitable for effective feedback of mobile robots is quite new, therefore we relied on related work from more general fields. Some previous studies examine the general psychological effect of color on people while others investigate also the practical utilization of light and color as methods of interaction between humans and computers or humans and robots. The following subsections discuss these topics in detail.

II-A The Meaning of Colors

Many studies have tried to explore the psychological effect of color on people. Various studies examine the relationship between colors and emotions. Terwogt and Hoeksma [1] examine the emotion-color linkages of three age groups from child to adult. Their evaluation shows that the linkages shift over time, e.g. seven year old children tend to relate blue with happiness, while adults tend to relate green with happiness. Their study also reveals that at all ages colors and emotions are strongly related. The same conclusion is drawn by Dael et al. [2] who investigate the linkage between color and the bodily expressions of elated joy and panic fear. Within their investigation, the users tend to choose colors of the red to yellow range of the hue encoding in case of elated joy and colors of the cyan to blue range in case of fear. In general, different studies with varying results imply that the association between colors and emotions is very heterogeneous. While Palmer et al. [3] reveal that red color tones are associated with both happiness and anger, other studies yield associations between blue and positive feelings [4, 5] as well as calm and sad emotions [3]. Naz and Helena [6] state that principle hues (red, yellow, blue etc.) are more connected to positive emotional responses than intermediate hues. Apart from general psychological aspects and interpersonal differences, there might be a contextual influence on color perception. For example, the interpretation of a red light on a TV is probably different from the interpretation of a red lamp on a car’s instrument panel or a synthetic light in a laboratory environment. Therefore, the following section reviews the related work regarding color and light in human-computer as well as human-robot interaction in order to account for this contextual influence.

II-B Colored Lights in Human-Computer and Human-Robot Interaction

Most devices of everyday use contain one or more lights that indicate information about the device. This includes devices all over the household like kitchen devices as well as multimedia devices or smartphones. Even though the use of lights is widespread, by far not all possibilities of this modality have been used, evaluated and studied yet. In order to simulate emotions, the field of affective computing examines the possibility to incorporate color and light in the interaction process to anthropomorphize computers. A review on the role of color and light in affective computing is given by Sokolova and Fernandes-Caballero [7]. Pradana et al. [8] study the possibility to prime emotions while reading incoming text messages using a ring-shaped color lighting wearable. Based on the assumption that there are positive and negative colors, the study shows that the use of such colors can shift the user’s emotional reaction on incoming messages. Song and Yamada [9] conducted a preliminary study that shows a positive effect of LED light animations on users during a computer game in terms of anthropomorphism and the user’s perception of the computer. While the use of color and light in affective computing is used to simulate emotions, the use of color and light to illustrate the status of a device is more common on a daily basis. An overview on how lights are used as a feedback modality in various technical devices is given by Harrison et al. [10]. Their study suggests that there are connections between light behaviors, e.g. blinking animations of LEDs, and the conveyed meaning. The systematic and empirically meaningful use of lights and colors to illustrate device states in robotics is rare. Sprute et al. [11] use colored light as a feedback mechanism in robotics. The current state of the system is illustrated using an LED strip mounted on the robot and using colored smart lights of the environment. Baraka et al. [12] recommend different color and light animation compositions for different robot scenarios, e.g. waiting, blocked and progress. In order to identify the best animation for each scenario, they carried out a study where experts and naive users were asked to give input on the animation type, speed and color. The result of this work was a list with a detailed description of the best animations for the introduced scenarios. Based on the previously mentioned study, Baraka et al. [13, 14] later investigate the positive effects of colored light on the understanding of and trust in robots by humans. They conducted a study where users hypothesize about the robot’s internal state in different robot scenarios supported by colored lights. In order to compare the different scenarios and to make a more universal statement, they categorize them in three distinct scenario classes. The study shows that the use of light has a significant effect on the accuracy of the users hypothesis as well as the general trust in robots. They claim that light signals are intuitively understandable, because humans generalize from other experience with devices that use light signals. Szafir et al. [15] propose different visual signaling mechanisms for flying robots using LED strips. They evaluate whether different light animations can help to enable users to deduce flight intents. The evaluation reveals that, while the participants believed that all animations help to make the robot a better partner, only a subset of animations enable participants to understand the robot’s light intents more quickly and accurate. A similar approach is proposed by Monajjemi et al. [16]. Betella et al. [17] conduct a study to find how affective states of technical devices can be expressed using colored motifs. The study reveals that the perception of affective states of technical devices is correlated to the color of the motif, e.g. pleasure is correlated to cool colors (long-wavelength hues). Collins et al. [18] illustrate the internal state of a robot using changing patterns of light based on the circumflex model [19]. The results of their study show that changing patterns of light can evoke reliable perceptions of affect in naive participants and can therefore convey complex information.

III Scenario and Approach

In our scenario, a mobile robot was able to interact and communicate with users using an attached LED strip shining in different colors. The mobile robot was located in a building and performed various scenarios. It was able to react to complex speech commands, to navigate on its own, to clean a room and was able to perceive objects in its environment. The study was designed as a Wizard of Oz experiment where the behavior was realized by simply remote controlling the robot. Real abilities were not necessary in our case, since participants only saw recorded videos of the robot.

The videos were recorded at four different locations in a university building: in front of an elevator (Fig. 1a), at a seating area in a laboratory (Fig. 1b), in a hallway (Fig. 1c) and in a kitchen (Fig. 1d). The LED strip is a fully-addressable WS2812-based RGB LED strip with 30 LEDs per meter. The strip is 12.5 mm wide, 4 mm thick and is connected to an Arduino UNO using a 2-pin JST SM connector. It is mounted on a TurtleBot2 with a white custom casing made of thin polystyrene in order to enable translucency (Fig. 2).

IV User Study

We conducted an experimental evaluation using an online survey to reveal the relationships between colored light and its semantics in the context of robotics. We assumed that the experience with colored light in daily life might influence our color semantics. This would also mean that the association is shaped by the degree and nature of the experience, which varies from person to person. This could lead to varying color semantics. Therefore, it was interesting to determine how experienced a user is. Our main question was whether there are stable prototypes of colors’ semantics that play a role in human-robot interaction or not. The study was conducted with a total of 56 participants, out of which 38 were male and 18 were female. Their ages ranged from 15 to 70 years (Mdn = 26, SD = 13.7), only one of them belonged to the group of elderly people (>60 years).

IV-A Experimental Design

The study was designed to reveal the relationship between colored light and its semantics in robotics. The participants saw videos of a mobile robot in different situations and were supposed to select the color that would help them to assess the situation and understand the state of the robot. In this context, it was assumed that every state of a robot is connected with an information the robot wants to transmit to the user. Following the work by Harrison et al. [10] and Baraka et al. [12], we identified three categories of states (information) that are general enough to cover several specific states. The three categories are Active, Help needed and Error, and they are shown in Table IV-A. According to Baraka et al. [12], we specified scenarios for each category to make them more tangible for our participants than abstract categories.