‘That’s just weird’: A qualitative investigation into expert opinions on the difference between autonomous vehicles and humans deciding to kill

TL;DR

Experts responsible for autonomous vehicles have complex views on whether AVs should make life-or-death decisions, highlighting ethical concerns and differences from human judgment.

Contribution

First qualitative study on expert ethical opinions regarding AVs making lethal decisions in collision scenarios.

Findings

Experts believe humans remain ultimately responsible for AV behavior in collisions.

AVs lack human traits like intuition and may make decisions that feel unnatural.

Some experts support AVs making life-or-death decisions in certain cases.

Abstract

Autonomous vehicles (AVs) are rapidly developing and in the process of being deployed on public roads. This has sparked extensive discussions on the ethics of AVs, particularly in collision scenarios. While much quantitative research has been done, little qualitative research has been conducted and none on the ethical opinions of experts who are actually responsible for developing, deploying, and regulating AVs on public roads. Making use of qualitative research methods, 46 experts were interviewed to obtain rich data on their ethical opinions of AVs deciding to kill human beings. Following thematic analysis, three overarching themes were identified: 1) Experts feel humans ultimately will be responsible for how AVs behave in collision scenarios. 2) AVs decisions lack important human characteristics such as ‘gut feelings, emotions, or intuition’ and would make uniformed decisions which do not reflect human decisions. 3) Some experts did have a preference for AVs making decisions in life and death situations. The paper ultimately concludes that experts who are responsible for how AVs are designed, deployed, and regulated hold complex opinions on the ethics of AVs making life and death decisions. Considering the public’s legitimate interest in this domain, far more work is needed to unify the ethical opinions of experts on the ethics of AVs in collision scenarios.

Figures

Full text

Introduction

The impact of Autonomous Vehicles (AVs) will be beyond simply transportation [1]. From improved traffic flow and congestion [2,3] to better general access for the elderly or disabled [4], from environmental benefits [5] to an impact on public health [6]. Yet without a doubt the greatest perceived benefit is the anticipated radical reduction of road deaths [4,7,8]. Some estimates put this at a 90% reduction [9–11]. Considering that annually over 1.3 million people die on public roads, with 50 million more injured, this would be an incredible benefit [12]. Yet it is this point precisely that has captured the public’s attention in paradoxical ways. How can AVs reduce road deaths ethically? More than reducing human error [8] and improving safety [13], AVs may need to take action in life and death scenarios that could present them with moral dilemmas?

While some dispute that AVs will ever be faced with these scenarios [14–17], others argue that it is inevitable [18–22]. Consequently, it is important that we wrestle with the ethics of trolley-problem-like scenarios for AVs [23], if for no other reason than the fact that the public is deeply concerned with these issues [24]. Indeed, the public’s engagement in the ethics of AVs choosing to kill or save lives is extensive [25] and has sparked widespread debate in the ethics of AV killing and in trolley problems in general [7,26–31]. The ethics of machines killing human beings have been debated long before AVs, particularly in the context of Lethal Autonomous Weapon Systems (LAWS) such as military drones. Some argue that these autonomous killing systems could be good news as they reduce human suffering [32], while others doubt the morality of killing at a distance [33]. There is reason to argue that the public are generally against machines making moral decisions [34] with the EU calling for a moratorium on LAWSs [35]. In the words of Coeckelbergh:

There seems to be a fundamental asymmetry between humans and robots when it comes to their capacities to make good moral decisions, and on this basis we should ban any project that aims to have machines (e.g., drones) make decisions about life and death [36].

For many, the question of autonomous killing is connected not only to LAWS, but also to AVs [36–38]: ‘After all, transport accidents, healthcare practices, and abuse of personal data may affect people’s life as much as military operations’ [37]. Indeed, critical perspectives have emphasized the importance of understanding AVs not as autonomous moral agents, but as components in a sociotechnical system shaped by human and institutional decisions [39–41]. Questions of transparency, responsibility, and value alignment cannot be reduced to algorithms or logic trees. Instead, as Martinho et al. [42] and Lapaolo [43] argue, we must examine how AV development is guided by ethical assumptions embedded within industry practices.

Despite the growing critical normative literature [15,44,45], there remains a notable lack of empirical research into the perspectives of those directly responsible for AV development, deployment, and regulation. While studies such as Stilgoe (2021) explore safety and risk governance [46], and Martinho et al. (2021) review ethical themes emphasized by the AV industry [42], few studies gather rich, open-ended insights from developers and regulators themselves. This gap is significant: as Stilgoe and Mladenović (2022) argue, the politics of AV development hinge on expert assumptions about ethics, autonomy, and public acceptability [41]. It is these experts who are both directly and indirectly responsible for how AVs behave in life and death situations. Assuming that experts are in a particular strong position to make informed decisions, their attitudes are vital to understanding how AVs might, or should, behave in situations which involve the life or death of members of the public. The present study is the first of its kind and goes some way to filling this research gap.

Methodology

Sample and data collection

As part of the National Centre for Competence in Research (automation – Switzerland), the Proactive Ethical Approach to Responsible Automation (PEpp) project interviewed 46 experts between the 21/03/2022 and the 27/07/2023 about the ethics of AVs on public roads and in particular on the ethics of AVs making life and death choices. Participants were chosen who had a range of expertise either directly or indirectly related to AVs on public roads at academic, private, and public organisations. A total of 33 experts had direct experience in developing AVs, this included developing motion planning algorithms, working for universities and private organisations to develop AVs, acting in senior leadership positions within AV organisations (such as CEO or Chief Engineer), testing AVs on public roads for well-known AV companies, or being involved in implementing public trials of AVs. Ten experts had indirect experience such as programming or researching in the field of traffic management, robotic and automatic control systems, or control engineering for transportation applications. Three experts were involved in the regulation of AVs in Switzerland, the EU, and the USA at state level.

Following ethics approval from the University of Basel’s Ethics Committee, a semi-structured interview guide was developed informed by the literature on AV ethics and aligned with the study aims. The semi-structured format allowed flexibility for participants to elaborate beyond the guide while ensuring comparability across interviews. Questions were designed to focus on the ethics of AVs on public roads. This included questions of the role and responsibilities of experts as well as how AVs should be regulated. Of significance for the results presented here are interview questions that explored experts’ opinions of AVs making life or death choices compared to human drivers making these same choices. Questions were open-ended, with optional probes and scenario-based prompts (e.g., trolley-problem-like cases) to elicit nuanced reflections. All participants gave their written as well as verbal consent before being interviewed. Initially three pilot interviews were conducted to test clarity, flow, and timing; these were evaluated by researchers and the PI, and the guide was refined accordingly.. Purposive as well as snowball recruiting was undertaken to identify potential participants [47,48]. Interviews were undertaken in English, lasting approximately 1 hour with participant recruitment continuing until saturation was achieved, upon which 2 further interviews were conducted to confirm saturation [49]. Using an experimental ethics approach to qualitative research, experts were encouraged to go beyond simply stating a belief they had about a particular topic, rather they were encouraged to actively engage in exploring and justifying their opinions [50–52].

Data analysis

All information permitting direct or indirect identification was removed from verbatim transcriptions of the interviews so as to anonymise all data (e.g., name, place of work, role etc.). Using MAXQDA – a well-established qualitative research analysis software program – transcriptions were subjected to reflective applied thematic analysis in order to analyse and highlight the significant thematic elements that arose from the data [53–55]. Researchers began by familiarising themselves with the data set through actively reading and re-reading the transcriptions while writing down initial ideas for codes and themes. Researchers proceeded to engage with open coding so as to inductively generate descriptive codes from the data. The first author initially coded three interviews after which a meeting was held with the research team and the PI to evaluate the quality of coding. Following this, the remaining transcriptions were coded with regular supervision and meeting with the PI. An extensive code tree was developed and agreed upon by the research team. Once the code tree was agreed, descriptive codes were sorted for themes. The research team reviewed the overarching themes created by the first author to agree the themes identified. In the last step the themes were interpreted and organised into coherent accounts that would support core narratives present in the data set. For more, see the companion articles reporting other findings on the research [56,57].

Results

This study recruited 46 experts who had direct and indirect experience of developing, deploying, and regulating AVs on public roads. Each participant engaged in a semi-structured interview based on an interview guide. Interviews were not prescriptive, and participants were encouraged to speak freely. The interviewees were sensitive to the leadings of the participants. Following data analysis, three key themes were identified. These themes (and their sub-themes) emerged from the data and often transversed individual questions and participants. The results presented here are of the participants who spoke to these themes (in some cases to multiple themes). As far as possible we present data without interpretation or discussions.

Is AI Actually Making Decisions or ‘Acting for Someone Else’?

1

A large number (16) of experts noted that fatal decisions enacted by AVs are not the result of AVs themselves but determined by external factors (see Table 1). This concept emerged throughout the interview process and was not mentioned by the remaining participants. Within the cohort of participants that mentioned this theme, a number of experts referred to human programmers/designers as the ultimate source of decisions enacted by AVs during collision scenarios. For example, Participant 3 explicitly stated: ‘decisions are not made by AI, they’re made by the designers of the AI.’ Likewise Participant 2 did not ‘truly see a difference’ between a human and an AV acting in collision scenarios.

Table 1: Theme 1 Data Extracts.

There was a sense among some participants that if AV decisions were merely the product of exogenous factors (including human programmers) then there is a question of who, or what, forms the basis for the ethics that guide decisions made by AVs in collision scenarios? For example, Participant 13 had ‘very big troubles understanding’ which human behaviour the AV would replicate. Participant 10 was unsure of who, or what morals, underpin an AV decision and, therefore, stated that they ‘would be against’ AVs killing humans in a dilemma scenario. By this they meant AVs should not be involved in the killing of human beings at all – although no clarification was given as to how that might work in practice.

Other participants in this group noted both human and non-human influences. Participant 8 argued that human ethical decisions are determined by the environment in which respective humans grow up – their upbringing, cultural and other ‘exogenous factors.’ AIs are likewise determined: ‘an artificial intelligence… also emerge[s] from some data or something that has been fed to this, to this intelligence.’ These participants felt that it was not the AV itself that determines the ethics involved in collision situations, but rather these exogenous factors (data provided, sensor equipment etc.). This environment naturally also includes humans. To Participant 6 ‘the AI is acting for someone else.’ Yet there was a tension in Participant 6’s position in that he/she also understood the AV as being neither ethical nor unethical in the same way that an animal acts neither morally nor amorally. To Participant 6, an AV killing a person could be compared to a lion killing a person: ‘you don’t see them [lions or AVs] as ethical necessarily.’

Should AVs Make Life and Death Decisions

2

The notion that AVs merely reflect exogenous factors (including human programmers) lead a number of experts to question the suitability of AVs having autonomy in ethically difficult scenarios such as road collisions. This larger theme emerged from a number of sub-themes associated with a wider concern about AVs making life and death decisions.

AVs Lack Key Human Characteristics.

2.1

Some participants expressed concerns about AVs making decisions that could ultimately kill a human being because these AVs lacked what some participants felt were important human characteristics necessary for such decisions (see Table 2). For example, ‘gut feeling, emotion or empathy’ (p#3). Participants 9, 22, 25, and 31 asked if an AI would have ‘some sort of moral compass that aligns with [humans]’ (p#9) but found that ‘difficult to image’ (p#9). Indeed having a moral compass in a machine, according to Participant 9, would be ‘just weird.’ Participant 22 argued that it would be impossible for an AI to make moral decisions because ‘it will take purely logic [sic]’ decisions; that is, decisions without emotion. They went on to argue that for these rational decisions to be perfect, the AV would need ‘the whole knowledge of the universe’ but this would never be possible ‘in this universe.’ We speak more about this in the next sub-theme.

Table 2: Theme 2.1 Data Extracts.

Uniformed Decisions are not Desirable.

2.2

While Participant 22 struggled to imagine a world in which an AV that was able to make a purely logical decision, others could imagine this with conflicting opinions on its value (see Table 3). Participant 5, linking to the idea that AV decisions ultimately come from human programmers, argued that programming AVs in dilemma scenarios would be good for society as it would ‘force us to really specify in code… what are our principles,’ and this would remove ‘any inconsistencies of human reasoning.’ While there were those who agreed that reflective thinking was better than reflexive thinking (see theme 3.1 below) others argued that a purely logical AV would make decisions that are ‘uniforminized [sic]’ (p#16). To Participant 17, ‘if we design an AI it is just the one algorithm. Every AV will do one thing.’ This would not be a ‘uniform distribution’– referring to the Bell curve – but ‘more like a point load distribution.’ Participant 17 felt we would lose something here as ‘everyone [humans] will have a different decision’ in such dilemma scenarios. Participant 17 saw this as a positive aspect of human drivers. To Participant 20, AVs that act in uniform ways, are acting ‘completely oppositely to human[s]…[who]have emotion.’

Table 3: Theme 2.2 Data Extracts.

Black Boxes.

2.3

A larger sub-theme, linked to the question of data transparency, was identified (see Table 4). Among the group of experts who expressed a more pessimistic view, i.e., they thought AVs should not make decisions in dilemma scenarios, a number of experts (6) expressed a concern about the transparency of AVs. This was connected with the theme of AVs not ultimately making their own decisions (theme 1 above). For example, Participant 3 stated that the opacity of AV decision making questions the foundations of those decisions, while Participant 18 argued that it would be hard to trace the decisions of an AV to see who ‘forced that decision’ – referring to the AV programmers and developers.

Table 4: Theme 2.3 Data Extracts.

Even among the group that felt AVs would be preferable to humans, a few participants only considered this possible if transparency could be guaranteed. For example, Participants 7 and 20 argued that if an AV decision-tree could be mapped, and we could know why a decision was taken, they ‘would actually prefer the AI’ (p#7) to a human driver. To Participant 8 this represented a ‘delegated’ decision which they would be happy with. Participant 11 put a further caveat on this criterion by arguing that the decisions of the AV must be shown to be ‘the best option compared to all other possible ones.’ Participant 11 did not elaborate on how the ‘best option’ would be measured.

Responsibility Must Remain with Humans.

2.4

Consequently, a number of participants felt it was important for humans to remain in the decision-making loop and ultimately take responsibility for the death of a human being (see Table 5). To Participant 4, even if the AV might make a ‘better’ (p#14) choice, ‘we are not even ready’ to accept this outcome, nor will we be in the future. While Participants 11 and 21 felt that humans should always be involved where moral or ethical decisions are being made – even if this is at the programming level. Participant 11 argued that even in other spheres where machines are used to kill people, such as military applications, ‘the end [is] screen[ed] by somebody human.’ I.e. the end result is evaluated by a human operator before being approved. Therefore, ‘there must be a human…that either approve the rule or approves the action’ (p#11) while Participant 21 argued that they would prefer if a user could take over at any time.

Table 5: Theme 2.4 Data Extracts.

A Preference for AV Decisions

3

Human ‘Reflex’ Decisions are Not Optimal.

3.1

A number of experts ‘prefer[ed] AI making the decisions’ (p#38) in ethically problematic scenarios if certain conditions were met (see Table 6). One aspect of this theme was that humans – under time constraints – do not make good decisions. Participant 5, for example, who initially stated that human programmers ultimately made AV decisions went on to argue that ‘in the future… the decisions made by an AI will probably be better than the decisions made in haste by a human under stress.’

Table 6: Theme 3.1 Data Extracts.

Participant 14 claimed that a person who ‘has no time to think’ and thereby makes an ‘instant reaction’ has not made an optimal decision. To them, AVs will make better decisions because they can make decisions faster than a human being. Participant 24 argued that humans make reflexive decisions which are not ‘real decisions,’ and that machines would have a better understanding of the world in dilemma scenarios because of their sensors and access to wider knowledge. While Participants 35 and 40 argued that an AV’s decision is a ‘vast improvement’ (p#35) on a human who gets tired and is not always vigilant. The participants who preferred AV decisions, however, acknowledge that one had to ‘assume’ (p#40) AVs were quicker than human beings, which they doubted were ‘true today’ (p#24) as ‘at the moment we are very far from that’ (p#14).

AV Decisions Offer Advantages.

3.2

There were two participants who highlighted the advantages of AV decisions (see Table 7). Participant 45 presented a paradox by arguing that an important ‘human psychological trait’ is to blame someone. Yet an AV could not be blamed as it was merely a machine. On the other hand they noted victims’ families – and the community as a whole – may better accept a fatal incident from an AI than a human because there is no ‘face to blame.’ For them, it would be easier to accept an AV killing a person, than another person doing the killing.

Table 7: Theme 3.2 Data Extracts.

Participant 39 argued that AVs give us the opportunity to immediately change road behaviours without having to retrain the entire population. For example, when we chose to reduce the road speed, or open a new school in an area. Changes to speed and behaviour will ‘be implemented on all the vehicles from tomorrow.’ Consequently, we can update the way life and death decisions are made instantly and do not have to wait for human beings to learn new ways of approaching challenging situations.

AVs Should be Held to a ‘Higher Standard’.

3.3

For Participant 29 the ability to make quicker decisions based on more information necessitated that AVs be held to ‘a higher standard’ when evaluating the ethics of their decisions (see Table 8). Similarly, Participant 5, argued that the pre-trained, rational response of an AV represents a form of ‘intent’ to kill that is not present in reflexive thinking.

Table 8: Theme 3.3 data Extracts.

Discussion

The results presented above paint a picture of non-uniformity among expert opinions. While many considered AVs to be merely acting out the instructions of human programmers, and therefore, not moral agents, others engaged with the idea of an AV making a life-or-death choice. For some, this was problematic as AVs lacked key human characteristics and would make non-transparent uniformed decisions that were ultimately decidedly different to how human beings make decisions. For others, enabling AVs to make decisions offered certain advantages, notably the reflective (as opposed to reflexive) nature of these decisions; the possibility for society to better come to terms with tragedy if it were the result of a machine, and the ability to immediately change road behaviours. Nevertheless, while it is possible that AV decisions should be preferenced, there was a strong sense that their decisions should be held to a high standard. Let us discuss each of these opinions in turn.

AVs are not moral agents

There is a challenge when speaking about an AV ‘choosing’ or ‘deciding’ to act. These terms are often associated with human beings and strongly connected to philosophical discussions around intentionality or moral agency [58,59]. Traditional algorithms cannot strictly be said to ‘choose’ or ‘decide’ a course of action in that they merely adhere to their programming. Modern AVs, however, are increasingly driven by AI which often acts as a black box with an opaque decision tree. During this research we deliberately did not define concepts such as ‘decision’ or ‘choosing’ as we were interested in the opinions of participants as to whether there is a difference between an AV’s actions and a human’s actions. Thus, we left it for participants themselves to interpret terms such as ‘choosing’ or ‘deciding’ when giving open-ended questions. The results of this research indicate that AV experts have divergent opinions of the challenges posed by AVs taking human lives compared with human beings making the same choices.

As expected some experts noted in theme 1 above raised questions about the nature of these decisions. That is to say, are such decisions actually the result of an AV itself, or merely of its programming and consequently those who have programmed the AV. Perhaps more problematic, some noted that the decisions may be the result of the training data or context of the transportation environment rather than a human programmer. While this was not explicitly stated, it is implicit among such experts that an AV is not itself an autonomous moral agent. A moral agent might be defined as an autonomous subject with a reasonably fixed idea of what is good and whose free will enables them to determine the principles of their own actions [7,60,61]. However, for many of our participants, AVs at present (driven by current AI as they are) cannot be understood as autonomous subjects. Therefore, it is incoherent to consider an AV making a ‘decision’ in the same way as a moral agent makes a decision. To the experts that spoke to theme 1, AVs remain merely machines that enact decisions based on exogenous factors. This is ultimately the current position of the EU on autonomous systems. The EU has stated that: ‘Since no smart artifact or system – however advanced and sophisticated – can in and by itself be called ‘autonomous’ in the original ethical sense, they cannot be accorded the moral standing of [a human person]’ [62].

In this sense, ethical implications of AVs are not reducible to instantaneous decision-making during a crash, but rather extend upstream to the institutional and technical infrastructures in which AVs are embedded. This aligns with the argument by Stilgoe and Mladenović (2022) [41], who emphasize the politics of AVs—that their development is never ethically neutral but reflects decisions about concern mundane issues of safety, accountability, and public trust that are made beyond an individual AV itself. Marres, for instance, shows how street trials of AVs are not merely technical tests but public experiments in participatory ethics, where nothing happening can be as meaningful as something going wrong [40]. Similarly, Ganesh critiques the “ironies of autonomy” in how the language of independence masks deeply relational and embedded ethical decisions [39]. These perspectives are echoed by several experts in our study, who emphasized that AVs are never truly autonomous but act within a framework of delegated human responsibility.

The lack of moral autonomy led some experts in this study to question the moral framework underpinning AV decisions that results in the death of a human being. This begs the question: Whose moral frameworks should be used as the guiding frameworks for how AVs make decisions? This discussion is taken up in the literature as a number of authors wrestle with the vague notion of AVs having to mimic human driving behaviour and ethics [38,63–65]. However, what is not stated in the literature is the notion that it is AV development experts themselves who are ultimately responsible for designing, programming, and implementing AV decision-making algorithms (DMAs). Some experts who participated in this study were keenly aware that AVs will not be making ethical decisions themselves but rather were heavily influenced by experts such themselves. As such they were conscious of the significant questions surrounding who exactly will be guiding these decisions. It should be noted that results reported elsewhere from this same study have shown that AV experts have poorly developed moral frameworks [57].

AV decision making is markedly different to human decision making

Theme 2 presented an interesting tension within some experts in this study that is worthy of note. Some of the experts who spoke to theme 2, also spoke to theme 1 (e.g., p#25 and p#9). On the one hand these experts understand AV decisions to be the direct consequence of exogenous factors, primarily of which is their human programmers. Yet on the other hand, these experts also expressed concern about the marked difference between an AV DMA and a human’s decision-making process. This concern implies that these experts believed that AVs are indeed making decisions and that these decisions are based on different processes/basis to those of a human being.

For the experts who spoke to theme 2.1 of this study, the apparent lack of human-like characteristics such as ‘gut feeling, emotion or empathy’ (p#3 also present in p#20) present in AV decisions was problematic for the ethics of AVs killing human beings. Reference to human-like values as the basis of ethical decision making is markedly different in our study to how it is presented in the literature. In the literature very rarely – if ever – does reference to such characteristics arise. Even among authors who argue that AVs should mimic human values, notions of ‘gut feelings’ emotions or human intuition are not the basis of these vales [38,65]. In fact, authors such as Frank et al. [63] would argue that such emotionality can lead to a bias in human values and presents a social challenge to producing a universal moral code for guiding AV DMAs. Rather than referencing characteristics such as emotion, much of the discussion in the literature revolves around higher-order reflective and rational ethics such as utilitarianism, deontology, or the social standing of one person versus another [25,63,65]. Even in discussions involving what has come to be known as the ‘ethical knob’ [66], whereby individual humans can choose what ethics to impose on their individual cars, the questions of what values should be included as options on this setting are often highly formal as opposed to practically rooted in human emotive characteristics [25,67–69].

Yet, the experts in our study, indicated that a lack of these apparent human characteristics would be a challenge to the ethical integrity of an AVs decision to kill a human being. This is in opposition to the views of authors such as Frank et al. [63]. That is to say, should AVs be considered responsible for their decisions, their lack of empathy or emotion would actually question the morality of these decisions. To add to the paradox, participants such as Participant 9 argued that AVs that did have these types of characteristics would be ‘weird’. For an expert such as Participant 9, an AV’s ethical decisions are ultimately merely programmed by a human being, yet they also argued that the AV’s decision-making process is devoid of certain human characteristics (‘empathy, gut-feeling, intuition, or emotion’) and if the AV ever did have such a human intuition, it would be ‘just weird.’

The paradoxes did not stop there. A key concern, expressed in the literature and the experts who took part in our study, is that of the black-box problem (sub-theme 2.3). This problem applies to AVs specifically and to AI generally. Transparency is fundamental to trust, yet AI often presents a challenge to this transparency [70–73]. We regularly cannot trace the decision-making processes made by an AI. The experts in this study stated that the decisions of the AV were ultimately those of human programmers and at the same time expressed concern that an AVs decisions were not transparent. That is to say, it is difficult to trace the decision-making process of the AV to identify how it made a specific decision, and this in itself is an ethical challenge according to some of the experts of this study. The experts in this study hid not seemed to reconcile these two apparent contradictions: that AV decisions stem directly from human programmers, yet an AVs DMPs are opaque and difficult to trace.

One notion that was significant for the topic of the ethics of AVs making life and death decisions, as opposed to human beings making these decisions, was uniformity (sub-theme 2.2). For some of our experts, uniformity in ethical decisions was not considered a desirable attribute. Some experts argued that there is something intrinsic about the non-uniformity of human decisions that adds to their moral integrity. The exact underlying assumptions are difficult to gauge. For some, like Participant 17, it would be against the normal distribution of ethical decisions (if such a thing were to exist) if all decisions were the same. To Participant 20, uniformity would be opposed to the types of human characteristics mentioned by them and others: such as empathy, gut-feeling, intuition, or emotion (sub-theme 2.1). The response from this participant is significant as it indicates that some experts transfer non-ethical frameworks to those aspects of their work that concern ethics. This may indicate a form of modified ethical deliberation. Participant 20 seems to indicate that heterogeneous decision making is an observed phenomenon that is ‘normal’ and desirable. Elsewhere we have shown that experts do not have a well-developed ethical framework [57]. In Participant 20 their non-ethical training (statistical analysis) is being used to dictate desirable behaviours for AVs in ethical situations. However, one would be hard pressed to find a trained ethicist who would approve of a normal distribution function as the basis for ethical decisions.

This aversion to uniformity is very rarely spoken of in the literature. On the contrary, Coeckelbergh [74], for example, notes that some may argue that a machine would make better moral decisions because they do not rely on emotions which, in moral philosophy, are held to be problematic because of their unpredictable nature. However, Coeckelbergh goes on to argue that a moral agent requires a certain type of reasoning (including feelings) for moral deliberation. Consequently, moral decision making requires a particular kind of knowledge ‘an embodied kind of knowledge’ [74] that is sensitive to contexts and situations, developed through practical wisdom and lived experience. While Coeckelbergh acknowledges the necessity for certain human characteristics, non-uniformity is not mentioned.

The experts presented in theme 2 of this study, however, emphasised both the ethical challenges to the inconsistencies of human reasoning, and the ethical value of non-uniformed outcomes. There is an implicit belief in much of the literature that consistent ethical decisions across AVs are desirable. From a technical perspective, consistency is often equated with fairness and safety. However, from the view of some experts of this study, human non-uniformity — our capacity for context-sensitive variation — is a moral asset. This insight parallels arguments by scholars like Marres [40] and Ganesh [39], who emphasize the value of ambiguity and local responsiveness in ethical governance. Yet there remains a clear paradox in arguing on the one hand that programmers are morally responsible for AI decisions, while also arguing against uniformity and in favour of emotion in making those decisions: by definition, programmers are not present ‘on the road’ and will (hopefully) endeavour to follow some type of objective and uniform ethical code or framework. How exactly one creates non-uniformed decisions without emotions or feelings is a challenge that cannot easily be resolved without relying on pure randomness – which itself would be ethically problematic.

Thus, experts who spoke to this theme of the study called for human beings to be ultimately in control of AVs when it came to life and death situations. While this need to exercise meaningful human control is well-documented in the literature [75], the basis for this control is markedly different in our study than in well-established frameworks such as in the EU. As we have noted, the EU statement on autonomous systems being involved in killing humans has argued that at present a machine cannot be a moral agent. However, this statement goes on to express that even if it is technically conceivable for a machine to be a moral agent, the basis of human rights – human dignity – ‘implies that meaningful human intervention and participation must be possible in matters that concern human beings and their environments’ [62]. In our study, however, the experts presented in theme 2 were concerned about an emotionless, opaque, uniformed process as the basis of deciding to kill a human being.

A preference for AV decisions

Among the experts who were of the view that AVs would make better decisions (theme 3), opinions were shared that both converge and diverge from the literature. The experts shared the well-known advantages of AIs generally; the ability to process substantial amounts of data and to make rapid decisions. To these experts, these facets of AI imply that an AV would make better decisions than human beings. What is not mentioned in the literature, at least to our knowledge, is the pastoral advantage of having an AV kill a human being. That is to say, that there is no ‘face to blame’ (p#45) in the tragedy of an AV taking a human life was a novel idea presented here. It is an open question as to whether this is indeed a pastoral advantage or not. Nevertheless, the perception held by Participant 45 was interesting and may well hold true for some.

Limitations

A number of limitations should be noted. Frist, this is an international study and many of the participants did not speak English as their first language. However, it should be born in mind that many of the non-English participants were engaged in highly technical activities and were therefore highly educated and working in primarily English environments. Thus we feel that this is not a significant limitation, and their meanings are accurately reflected in this paper.

A second limitation is that the participants of this study are primarily from, or working in, Western contexts, mainly in Europe and the USA. Thus the opinions and ideas expressed by participants come from a particular cultural perspective. Consequently, it would be difficult to extrapolate from this data the opinions of experts from other regions or cultural groupings. While the data may not be generalisable, this does not diminish its significance. In fact, as with the methodology employed (qualitative research) the ultimate goal is not to produce generalisable results, but to provide rich descriptions of a phenomenon, in this case the opinions of experts on AVs making life or death decisions in comparison to human drivers making these same decisions [76,77].

Conclusions

AVs present numerous ethical challenges one of which is the ethics of autonomous killing. While this has been debated in other fields, such as LAWS, it is also pertinent to AVs. There is evidence that the public is interested and concerned with this aspect of AVS and if AVs are to be widely adopted, the ethical concerns must be addressed. While there has been normative and empirical research conducted in this field, no research has considered the ethical opinions of those who actually regulate, develop, and deploy AVs on public roads. This is a noteworthy gap in the research, as it is these experts who are directly and indirectly responsible for how AVs behave in life and death scenarios. Consequently, the present research project, and the findings presented here, are significant. The findings of this qualitative research demonstrate that experts hold a diversity of perspectives. While some understand AV decisions to be nothing more than the consequence of exogenous factors (including human programmers), others do indeed perceive AVs making decisions and that these decisions are fundamentally distinct from the decisions taken by human drivers.

Some experts are of the opinion that AVs would make better decisions than human drivers in life and death situations. Unlike human drivers, AV decisions would be non-reflexive. They would be deliberate, intentional, logical, and thought through. Furthermore, it might be easier for the human community to come to terms with the consequences of a life and death decision made by a machine because it would be difficult to blame a machine in the same way one blames a human being. Nevertheless, these decisions need to be held to a higher standard than human decisions.

On the other hand, many experts expressed concerns about AVs making life and death decisions for a number of reasons. AVs apparently lack key human characteristics such as feelings, emotions, or empathy, and would make purely logical decisions. While the decisions may be based on pure logic, according to participants who question AVs ability to make better decisions, this itself presents challenges. First, the transparency of AV decisions is still a key ethical issue as the ‘black box’ nature of AI is well-known to those developing this technology. Second, the uniformed nature of AV decisions is problematic. To many of the participants of this study, there is something of value in the non-uniformed nature of human decisions. That is to say, the breadth of human responses is a valued aspect of human drivers.

However, it should be born in mind that many participants of this study felt that any decision made by an AV would ultimately reflect the decisions of human developers. To these participants, the AV is merely a tool and, therefore, could not be held accountable for its decisions. AVs do not represent moral agents. For many participants this was simply the nature of AVs, while for others this was a desired reality. Those that questioned the morality of a machine making life and death decisions, expressed a strong desire to retain a high level of meaningful human control [75]. This, however, presented a paradox as participants desired non-uniformity in decisions, but also uniformed control.

This research has demonstrated that those who are responsible for the regulation, development, and deployment of AVs on public roads do not have a unified opinion on the ethics of AVs making life and death decisions. While some feel that AVs will make better decisions, for most of the experts in this research there were fundamental questions about the ethics of allowing an AV to make these decisions, the types of decisions AVs will make, and the role of human beings in influencing these decisions. Considering the publics’ legitimate interest in this domain, far more work is necessary on the ethics of AV decisions in life and deal scenarios.