Medical rationing choices of laypeople and clinicians are often illogical and inconsistent with their own stated preferences

TL;DR

During the pandemic, both laypeople and clinicians made inconsistent and illogical decisions when choosing who should receive limited medical resources.

Contribution

The study empirically demonstrates inconsistencies in rationing decisions influenced by irrelevant options and personal policy contradictions.

Findings

Adding an irrelevant third patient systematically altered choices between two patients for treatment.

Decisions conflicted with participants' own stated rationing policies.

Results support the need for predetermined, fair policies in medical rationing.

Abstract

The sudden onset, rapid spread, and later surges of the Covid-19 pandemic resulted in shortages of ventilators, pharmaceuticals, and other critical resources, leaving individual clinicians to make rationing decisions for which they had little expertise or training. In two pre-registered experiments with large samples of laypeople (N = 2007) and clinicians (N = 1256), conducted during the first year of the pandemic, we found evidence of two inconsistencies in hypothetical rationing decisions: (1) The choice of which of two patients should receive a medical treatment can be systematically affected by adding a third patient who logically should not receive the treatment (an instance of the attraction effect); (2) Decisions as to which patient should receive the treatment are inconsistent with general rationing policies that participants themselves endorse. We argue that our results provide empirical support for the necessity of predetermined policies administered by independent decision-makers to ensure fairness and consistency, as required by law and ethics, in healthcare rationing choices.

Figures

Panel A: Attributes of patients in the ventilator rationing experiments (the patients in the remdesivir experiment were one unit higher in every attribute).Demonstration of the attraction effect in each experiment for laypeople (Panel B) and clinicians (Panel C)–Patient B is chosen more frequently in the ABD condition than in the AB condition.

Full text

1 Introduction

When there are more patients who may benefit from medical treatment than there are human or material resources to provide that care, treatment must inevitably be rationed. Rationing of potentially life-saving resources such as intensive care unit (ICU) beds is not new [1]. However, the sudden onset, rapid spread, and later surges of the Covid-19 pandemic disrupted the global economy, creating shortages in many industries including healthcare. This shortage resulted around the world in periodic rationing of several resources for both Covid-19 and other patients, including ICU beds, ventilators, extracorporeal membrane oxygenation (ECMO) machines, pharmaceuticals, and staff who perform surgeries and other critical procedures—often without formal guidance [2]. For example, in February 2020, frontline doctors in Northern Italy reported substantial distress from having had to make life and death, bedside decisions regarding who would and would not receive care [3,4]. To protect frontline clinicians from the moral and potentially legal responsibility for rationing [5,6], and to ensure consistent, equitable decisions [7], both states and institutions quickly developed algorithms for allocating scarce healthcare resources, in most cases to be implemented by triage officers or committees with no clinical responsibilities for individual patients. By May 2020, more than half of U.S. states had publicly available guidelines for allocating ventilators, although criteria differed substantially [8]. States that were hit hardest by the pandemic eventually invoked those standards, sometimes extending rationing to non-Covid-19 patients [9]. Yet emerging evidence from the second year of the pandemic, when crisis standards of care were most commonly activated [10], suggests that even clinicians who practiced in states or institutions with formal guidelines ultimately made “in-the-moment” rationing decisions on their own, largely detached from those guidelines, and that bypassing frontline clinicians in resource allocation may be “unworkable” [11]. It is therefore important to better understand individual clinician decision-making in this context.

Research in psychology and economics has consistently demonstrated that when people are confronted with specific decisions– such as which consumer product to purchase or which medical treatment to prescribe (in case of doctors)– they can make choices across cases that may be inconsistent with one another [12–15], with normative decision rules, or with a general policy that they endorse [16]. Experiments have shown that choices can be influenced by factors that are not logically relevant, as well as by anxiety and other short-term mental states [17]. Thus, in-the-moment rationing decisions made by clinicians might be subject to the same biases, in which case Covid-19 treatment allocations may not be optimal and might not align with social preferences and policy goals—or even decision-makers’ goals.

In a set of two preregistered studies, one with members of the general public (2007 U.S. crowd workers across two online platforms) and the other with professional clinicians (1256 doctors, other prescribing clinicians, and nurses employed by a large U.S. health system), we explored whether and how decisions about Covid-19 treatment rationing might be subject to context effects, how often a person’s decisions reflected their own preferred rationing policy, and whether making such choices (even in our hypothetical conditions) caused deciders to report stress or any other discomfort.

We hypothesized that the choice of which of multiple Covid-19 patients should receive the only ventilator or drug dose (remdesivir) available will be influenced by one or both of two well-studied context effects on multi-attribute choice: the attraction effect [13], pre-registered as a confirmatory hypothesis, and the compromise effect [14], pre-registered as an exploratory hypothesis. The attraction effect occurs when adding a dominated option D to a set of two options A and B “attracts” choosers to the option that dominates D (say, B), and thus increases their likelihood of choosing B. The compromise effect occurs when the addition of an option E that makes a previous option B now a “compromise” between the new option and the other previous option A increases the choice likelihood of the compromise option (B) relative to option A.

We also evaluated participants’ preferences over treatment rationing policies in two ways: first, we asked them to rate 10 different rationing policies and choose their favorite. Second, we asked if 12 specific factors should be considered in rationing decisions. We used participants’ responses to these questions to explore whether their own rationing choices were consistent with their endorsed policies. We also investigated whether participants demonstrated any self-serving biases in their decisions, i.e., favoring (disfavoring) policies or factors that would serve themselves better (worse) as patients.

Previous studies have demonstrated the presence of both the attraction and compromise effects in various domains, including in consumer products [18–22], monetary lotteries [23], political candidates [24], and policies [25]. However, their relevance for medical decision making, particularly in relation to rationing decisions, has not been investigated to the best of our knowledge. Although Redelmeier and Shafir [15] study context effects in medical decision making, our study is novel in three key aspects: 1) The items of choice in [15] are treatments rather than patients; 2) [15] study other context effects, namely the similarity effect and status quo bias, rather than the attraction and compromise effects. That is, our work concerns completely different ways in which decision-making may depart from normative standards; 3) We also directly consider how decisions may conflict with decision rules or preferences that the same participants endorse, which [15] do not measure.

2 Materials and methods

Participants. Laypeople (n = 2007, restricted to the U.S.) in Study 1 were crowd workers recruited via two online platforms: Amazon Mechanical Turk (MTurk) and Prolific, with each platform constituting half of our total sample size. With each platform, a single Qualtrics link was used that randomly assigned around 500 participants to each of the two rationing experiments (ventilator or remdesivir). Moreover, within each experiment, Qualtrics was set to evenly and randomly distribute participants across the three conditions (AB, ABD, ABE) in that experiment. All of these participants were paid $1. Both experiments were launched and completed on 08/27/2020. The experiments were advertised as surveys asking for participants’ opinions on an issue of current importance.

Clinicians (n=1256) in Study 2 were doctors, nurses, and other healthcare providers who worked for a large health system in the northeastern United States. These clinicians participated in our survey by responding voluntarily to an email that invited them to take the survey with the prospect of winning a $50 Amazon gift card. This experiment was launched on 01/25/2021, and data collection was closed on 02/13/2021. This experiment was the first part of a two-part survey where the second part was completely unrelated to this study. To reduce selection bias as much as possible, we kept the recruitment email for the survey fairly general and advertised it as a brief survey of the opinions of clinicians regarding possible decisions about treating and studying COVID-19. The ambiguous nature of the recruitment email regarding the content of the study (rationing) as well as the monetary incentive for filling out the survey have potentially reduced the bias in recruiting more prosocial people, an attribute which might have correlation with rationing decisions.

Although these studies were exempt from IRB review under the Common Rule and its requirement of informed consent, we disclosed to prospective participants the main details of the study, including its purpose, and what participants would do and be paid. Participants could quit at any time by closing their internet browser window, and those who completed the survey were deemed to have consented.

Study format. The experiments with laypeople (ventilator and remdesivir versions) started with a scenario where multiple Covid-19 patients needed the medical treatment, but there was only enough of that treatment available for one patient. Participants were asked to choose the patient whom they thought should receive the medical treatment. They were then asked to explain why they had chosen the patient that they had chosen in the previous question. Participants then proceeded to answer two attention/robustness check questions which were similar to the first question of the survey except that among the two patients that were available in each of those questions, one was dominated by the other. The experiments then proceeded by evaluating participants rationing policy preferences in a set of three questions. Afterwards, participants were given the Covid Anxiety Scale (CAS) questionnaire [26]. Finally, several demographic information of participants (age, sex, educational attainment, income, political ideology, political party, and religion) were collected at the end of the experiments.

The experiment with clinicians started with the same scenario as the ventilator version of the experiment with the laypeople and followed with the same attention checks and policy preference questions. In this experiment, however, clinicians were not given the CAS questionnaire. We also collected clinicians’ sex and age, as well as their medical position, the number of years working in the medical field, field of specialty, and level of familiarity with scientific research methods.

In both the ventilator and remdesivir rationing scenarios, laypeople were presented with an introductory text that describes the corresponding medical treatment and its role in treating Covid-19 patients. The text further highlighted the scarcity of these treatments, emphasizing that not every patient requiring them may be able to receive them, hence necessitating the implementation of rationing measures. In the introduction of the ventilator scenario presented to health care workers (they did not see a remdesivir scenario), participants were specifically instructed to base their answers on their own personal opinions rather than the policies of their own institution or any other institution.

In both the lay and health care worker experiments, participants were first presented with a situation in which multiple Covid-19 patients need the same scarce medical treatment. Participants were told that based on the patients’ medical records, a computer algorithm had evaluated them in the following two attributes: (1) the likelihood of surviving upon receiving the treatment (expressed as a percentage) and (2) the anticipated years they would be expected to live if they survive Covid-19 (life expectancy). Participants were then asked to decide which patient should receive the only available ventilator (or dose of remdesivir). Note that computer algorithms are frequently used in healthcare and are becoming more prevalent in various tasks (See, for instance, [27]).

To ensure equalized need among the patients in the ventilator scenario, we explicitly stated that each patient would definitely perish if they did not receive a ventilator. In the remdesivir scenario, we sought to achieve a similar balance by stating that all patients met the criteria for the drug as per the government’s decision to approve it for emergency use in treating Covid-19. Furthermore, it was emphasized that all patients were expected to experience a better outcome if they receive remdesivir compared to not receiving the drug.

Both experiments employed a between-subjects design in which each participant was randomly assigned to one of three conditions: AB, ABD, and ABE, each representing the set of two or three patients in need of the medical treatment. In the experiment with clinicians, we dropped condition ABE and only used conditions AB and ABD. Patients in the ventilator scenario are described in Table 1 and demonstrated in the graph on the left panel of Figure 1. For the remdesivir scenario, we increased the value of each attribute of these patients by one unit in order to evaluate the robustness of our results against small changes in the attribute values.

Panel A: Attributes of patients in the ventilator rationing experiments (the patients in the remdesivir experiment were one unit higher in every attribute).Demonstration of the attraction effect in each experiment for laypeople (Panel B) and clinicians (Panel C)–Patient B is chosen more frequently in the ABD condition than in the AB condition.

Table 1: The list of patients in the ventilator experiment. Chance of Survival is conditional on receiving the ventilator. Life Expectancy is conditional on surviving Covid-19

We used R to do all statistical analyses. All survey materials, replication data, code, and preregisterations are available on OSF via the following link: https://osf.io/dgv3z/?view_only=81975a0062f145be9b1713b325ed786b+.

3 Results

Table 2 displays the percentage of participants who chose each patient, along with the sample size for each condition, across both scenarios.

Table 2: Choice shares of patients in different conditions and experiments. Each label in the Condition column refers to the labels of the patients that were presented in that condition. For example, in condition AB, patients A and B were presented.

As anticipated, choices made by laypeople revealed an attraction effect. Specifically, the choice share for Patient B increased from 39% to 45% in moving from the AB condition to the ABD condition ( $[eqn]$ ). This overall effect was driven by the ventilator rationing experiment, in which the choice share of Patient B increased from 38% to 48% when moving from the AB condition to the ABD condition ( $[eqn]$ ). We did not observe an attraction effect in the remdesivir rationing experiment (χ2=0.041,df=1,p=.83).

In the experiment with clinicians, we also found support for our confirmatory hypothesis concerning the attraction effect; the choice share of Patient B increased from 37% in the AB condition to 45% in the ABD condition ( $[eqn]$ ). See Figure 1 for the visualization of the patients attributes and the choice shares of patient B in different conditions.

To evaluate our exploratory hypothesis, the compromise effect, which was only tested in the laypeople experiment, we compared the choice share of Patient B in condition AB (i.e., $[eqn]$ ) with the choice share of Patient B in condition ABE when divided by the sum of the choice shares of Patient A and Patient B in condition ABE (i.e., $[eqn]$ ). This is equivalent to the comparison of the choice share of Patient B relative to the choice share of Patient A across the two conditions. This hypothesis was not supported in either scenario nor combined (Combined: $[eqn]$ in AB $[eqn]$ , $[eqn]$ in ABE $[eqn]$ , $[eqn]$ , Ventilator: $[eqn]$ in AB $[eqn]$ , $[eqn]$ in ABE $[eqn]$ , $[eqn]$ , Remdesivir: $[eqn]$ in AB $[eqn]$ , $[eqn]$ in ABE $[eqn]$ , $[eqn]$ ).

To screen out inattentive participants and/or those who did not find one or both of the attributes that we used to describe patients relevant for their decisions or used them in the opposite order intended (e.g., preferring older patients and inferring patients with higher life expectancies must be older), we included two robustness check questions. The scenario described in these questions was similar to the rationing scenarios outlined above. Specifically, in each question there were two patients who needed the same scarce medical treatment. In robustness check question 1, these patients had equal chances of survival upon receiving the treatment, but one had a higher life expectancy if they survived Covid-19. In robustness check question 2, the two patients had equal life expectancies if they survived, but one of them had a higher chance of survival with the treatment. Participants who selected the dominated patient in either of these questions were either inattentive or did not consider the attribute in which the dominating patient was superior to the dominated patient as relevant to their decision. Consequently, it would be appropriate to exclude these participants from the analysis of this question. In the laypeople and clinicians experiments, 2% and 3% of the participants failed these checks, respectively. After excluding these participants, the statistical significance of our results remained unchanged. However, both the attraction effect and compromise effect slightly intensified in the lay people experiments, while the attraction effect dropped by 1% in the clinicians experiment (for the details of this analysis, see supporting information).

The second part of all our experiments involved posing three questions to our participants regarding policies designed for rationing the corresponding medical treatment in situations akin to those described in the questions where they had to choose a patient directly.

In policy question 1, we presented participants with ten different rationing policies and asked them to rate their degree of approval of each policy on a five-point scale ranging from strongly disapprove to strongly approve, with a neutral midpoint. The table below presents the names of the ten policies as presented to participants in the ventilator rationing experiment (in the remdesivir experiment, the term “ventilator” is replaced by “remdesivir”). It is important to note that participants were only provided with the description of these policies during the experiment, while the names are only included here for clarity. The exact descriptions of these policies can be found in Table 3.

Table 3: Policies that participants were asked to rate and pick.

In policy question 2, participants were asked to select which of these (or an “other” policy) they believed a hospital should implement. The Spearman rank correlation between the appropriateness ratings of the policies and the percentage of participants who picked them as their favorite policies is $[eqn]$ . Table 4 provides the results of policy questions 1 and 2. Table 12 in the supporting information reports the results of these questions in the laypeople experiment separately for each scenario; ventilator and remdesivir.

Table 4: Approval ratings of policies and their choice shares.

In policy question 3, we presented different pieces of information about Covid-19 patients which could potentially be considered in rationing policies and asked participants to tell us whether in their opinion each piece of information should or should not be considered in rationing situations. Table 5 presents the pieces of information (for the remdesivir experiment, we replaced every instance of ventilator with remdesivir) and their descriptions. Table 6 presents the percentage of participants who indicated each piece of information should be considered in a rationing policy. Note that like the policies, we only gave participants the description of each piece of information. The names used here are chosen for the ease of exposition.

Table 5: Pieces of information that were presented to participants.

Table 6: Percentage of participants who have indicated that each piece of information about the patients should be considered in a rationing policy.

Within this study, some of the policies and pieces of information we included are actually implemented in real-world scenarios for rationing scarce Covid-19 treatments. Specifically, the Hybrid policy is very similar to the Pitt-Penn model hospital policy for rationing scarce medical resources (for details of this policy, see pages 6–9 of the document in the following link: https://www.ccm.pitt.edu/sites/default/files/202305/Allocation_of_Critical_Care_in_Public_Health_Emergency_June2021-FINAL.pdf). Additionally, the neighborhood factor corresponds to one of the primary considerations in the Pittsburgh policy for rationing remdesivir. Therefore, our results on these policies and factors can be potentially important. In fact, the Hybrid policy emerged as the most favored policy. However, the Neighborhood policy, which grants preference to individuals residing in disadvantaged areas, received significant disapproval, and moreover, few of the participants believed that people who come from economically disadvantaged neighborhoods should be treated differently. On the other hand, the First-Come, First-Served policy and the Hospitalization Time factor, which are not used in most real-world policies and are often criticized for their perceived unfairness toward minorities or underserved individuals, were rather popular among our participants.

Rationality demands decisions to be made according to some fixed principles or rationales, which implies that decisions that are made in similar domains should be consistent with one another. As such, in the context of our studies, rationality requires that participants’ chosen patient in the first question of the survey be compatible with their responses to the policy questions. Among the 10 policies above, the Survival Chance Maximizer, Life Expectancy Maximizer, and Utilitarian policies make unambiguous recommendations as to which patient should receive the medical treatment in each of the three experimental conditions we presented (AB, ABD, and ABE). The Survival Chance Maximizer policy recommends Patient A in all three conditions, as this patient has the highest chance of surviving among all patients if they receive the medical treatment. The Life Expectancy Maximizer policy recommends Patient B in the AB and ABD conditions and Patient E in condition ABE, since Patient B and Patient E have the highest life expectancy conditional on surviving Covid-19 in those conditions. The Utilitarian policy recommends Patient B in conditions AB and ABD and Patient E in condition ABE, as these patients achieve the highest score on the product of chance of survival and life expectancy in those conditions.

To assess the prevalence of irrational behavior based on the aforementioned measure of rationality, in our second main analysis we calculated the proportion of participants who chose any of the three policies mentioned above, but whose chosen patient did not align with their chosen policy in their corresponding experimental condition. Overall, among the experiments involving laypeople, 43% of the 770 participants who favored either the Survival Chance Maximizer policy, the Life Expectancy Maximizer policy (which both incorporate only one factor about patients), or the Utilitarian policy as their favorite policy, chose patients that were not consistent with their favorite policy. This percentage slightly increased to 44% (out of 733) after we excluded participants who had chosen a dominated patient in one of the robustness check questions.

In the experiment with clinicians, out of 494 participants whose favorite policies were among the above three policies, 44% chose patients that were inconsistent with their favorite policies. After excluding people who selected a dominated patient in either of the robustness check questions, we ended up with 488 participants who favored one of the mentioned three policies. Even within this group, 43% still selected patients that were inconsistent with their favorite policies. Among these three policies, the Utilitarian policy was both harder to think of and more complex to follow. Therefore, the described inconsistency between the choices of participants who chose this policy might be considered less “irrational.” However, even after focusing only on the 437 participants in the laypeople experiment who chose the Survival Chance Maximizer or Life Expectancy Maximizer policies, we found that 31% had chosen patients who would not have been selected by their preferred rationing policy. This percentage stayed the same after we excluded participants who failed the attention checks. In the experiments with clinicians, focusing on participants who chose one of Survival Chance Maximizer or Life Expectancy Maximizer policies, the percentage dropped to 27% out of 206. This percentage stayed the same after excluding inattentive participants. These percentages are robust to excluding participants who were in the bottom 10th percentile in terms of the time they took to finish the experiment.

To test the robustness of these results even further, we redid the consistency analysis by excluding participants who either chose a dominated patient in the robustness check questions or whose selected policies were not among their highest rated policies, as measured in the policy appropriateness question. In the laypeople experiment, we ended up with 1392 participants. In this subsample, 556 chose one of the three relevant policies, 38% of whom chose patients inconsistent with their selected policies. In the experiment with clinicians, the described exclusion resulted in 1044 remaining participants. 459 of these participants chose one of the three relevant policies, out of which 44% chose a patient that was inconsistent with their selected policies.

In the experiments with laypeople, we also explored the relationship between several demographic variables of participants, as well as their scores on the Covid Anxiety Scale (CAS) [26] in relation to their patient choices and policy preferences. The detailed results of these analyses are presented in supporting information. However, of particular relevance to the aims of our studies is the impact of demographic variables that directly affect how participants would be treated under our rationing policies. For example, younger people would generally benefit from the Young policy while older people would be at a disadvantage. A similar story applies to the Hybrid policy due to the preference it gives to younger people. Therefore, a higher support for these policies among younger people can be interpreted as a self-serving bias, which goes against the idea of impartiality, which is a quality that fair rationing policies should satisfy. Consistent with this bias, in a regression of policy approval ratings on the demographics of participants, we found a significant negative relationship between age of the participants and their ratings of the Hybrid policy (estimated coefficient −0.01,p=0.001) and Young policy (estimated coefficient $[eqn]$ ). Also, in a logistic regression of whether or not participants thought each piece of information about Covid-19 patients should be considered in a rationing policy on the demographics of participants, we found a significant negative relationship between the age of the participants and whether they thought a patient’s age should be considered in a rationing policy in policy question 3 (estimated coefficient $[eqn]$ ).

Given the correlation between life expectancy of a patient and their age and the positive bias of older participants towards older patients, it is plausible that older participants might be more likely to approve using life expectancy in a policy or favor the life expectancy maximizer policy. We examined these relationships. The estimated effect of age on the probability of approving life expectancy as a piece of information in a policy is -0.0118 and statistically significant (p = 0.020; see page 7 in the appendix). However, the relationship between age and approval of the life expectancy maximizer as a policy itself is negative but not statistically significant (p = 0.296).

See supporting information for the details of robustness check and demographic analyses.

4 Discussion

In two preregistered experiments with the participation of over 3000 laypeople and clinicians, we investigated the prevalence of two well-known choice anomalies–the attraction effect and the compromise effect—in medical rationing decisions. Additionally, we explored whether participants’ rationing decisions were aligned with their own preferred rationing policies.

In both lay and professional respondents, we found evidence for the attraction effect in ventilator allocation. This is significant, since the addition of a patient who, compared to another patient, is a less appropriate recipient in every described attribute (chance of survival conditional on receiving the ventilator, and life expectancy conditional on survival of Covid-19), and is thus an “irrelevant” alternative, nevertheless affects who among the remaining recipients will receive the ventilator. The replication of the attraction effect in the ventilator domain across two different populations (laypeople and clinicians) points to the robustness of the attraction effect in this domain. Given the importance of ventilators in treating a wide variety of clinical conditions in addition to Covid-19, this result has potential practical relevance.

However, we did not find evidence for the attraction effect in the remdesivir scenario with laypeople (and we did not test it with the clinicians). Moreover, we did not find support for the compromise effect in either of the ventilator or remdesivir scenarios among laypeople, the population in which the effect was tested.

Given the similarity of the ventilator and remdesivir scenarios, it is puzzling that the attraction effect was observed in the former but not in the latter scenario. In what follows, we present a few arguments that might explain this contrast. First, despite their similarity, there was an important difference between the two scenarios. Participants in the ventilator scenario were told that patients not served with the treatment would certainly die, but this explanation was absent in the remdesivir scenario. The higher stakes in the ventilator scenario are likely to elicit stronger emotional and cognitive responses, including a greater reliance on dominance as a justification for choice. This, in turn, may amplify the observed attraction effect. In support of this interpretation, [14] demonstrated that requiring participants to justify their choices to others increases the strength of the attraction effect. Similarly, [28] show that binding or incentivized choices, which involve higher stakes, tend to enhance this effect. Together, these findings suggest that more consequential decisions, like those in the ventilator scenario, may produce a stronger attraction effect. Nonetheless, we acknowledge that this explanation remains speculative and should be explored with more care in future research. Second, the absence of a significant effect in the remdesivir scenario may be partially attributed to noisier data and lower statistical power. Less consequential decisions, as in the remdesivir scenario, may reduce participants’ engagement or attention, increasing variability in responses. Additionally, there are indications that the attraction effect was observed in the Prolific portion of the sample but diminished when combined with the MTurk sample, which is known for higher variability and more inattentive responding. Specifically, we found a statistically significant attraction effect in the remdesivir scenario among the participants who were recruited via Prolific (half of our sample) after the exclusion of people who failed the attention checks. In particular, 54 out of 154 participants (35%) in the AB condition and 72 out of 148 participants (49%) in the ABD condition chose Patient B, giving an attraction effect of 14% ( $[eqn]$ ). This effect was offset by the absence of an attraction effect among the MTurk participants.

It is important to emphasize that the attraction effect observed in the ventilator scenario was both preregistered and replicated across distinct cohorts, including lay participants and medical practitioners. This replication provides strong evidence for the reliability of the findings in this domain, despite the overall null results observed in the remdesivir scenario.

Analyzing the policy preferences of participants, we found a substantial degree of inconsistency between how participants allocate treatments to the patients before them versus the rationing policy that they endorse the most. Specifically, of the participants whose preferred rationing policies unambiguously recommend an appropriate patient, more than 40% in both experiments choose different patients.

Random errors and deviations from true preferences in one or both of the patient choice and the policy choice questions can lead to inconsistent answers across the two questions. However, such errors are unlikely to be a significant source of inconsistency in our data, since very few participants made an error in the attention check questions (i.e., chose a dominated patient), questions that were very similar to the main patient choice question. Also, our results remain relatively robust after excluding participants who did not pass the attention checks, those who finished the experiment relatively faster, and those who did not select a policy that was one of their highest rated policies. Moreover, given the similarity of the degree of inconsistency in both experiments, it is unlikely that our results are due to sheer random errors. Even if such errors are partially responsible for the observed inconsistency in our data, they must be avoided in actual rationing decisions. One way to avoid such errors is following predetermined rationing policies.

Another explanation for the described inconsistency, which we believe is more likely, is that when deciding which patient should receive the treatment, participants do not necessarily summon pre-defined preferences or policies to make a decision. Rather, they construct their preferences in the given context based on the available options (patients). This idea is consistent with the literature on context-dependent preferences ([29]). Now, when a participant’s revealed preference (choice) in the patient choice question is a function of the available patients, it may conflict with a universal (context-free) policy that they get to endorse later in the experiment.

This explanation suggests that if participants were able to think about their favorite policy prior to the patient choice question, their inconsistency would be reduced. Exploring this idea can be an interesting avenue for future research.

This finding may also be a manifestation of the documented gap in people’s behavior when they are in “hot” versus “cold” mental states (see [30] for a review). In particular, when people judge policies that are to be applied to unidentified people in the future, they may operate in a “cold” and “rational” frame of mind. However, when confronted with the more emotionally charged decision of deciding which individual patient should receive a treatment, they may enter into a “hot” frame in which they summon a different set of rationalizations and heuristics, hence the inconsistent behavior. This result complements the result of [31] who find a discrepancy between the rate with which their participants ration medical treatments at the bedside and policy levels in hypothetical rationing scenarios.This result may also be framed in terms of the documented discrepancy between the way doctors recommend treatment to individual patients versus groups of patients [32,33] as well as the discrepancy between people’s inferred utility functions and their rationing decisions [34]. Similar to our work in spirit, [35] also compared stated principles and stated preferences in a related context, and [36] considered the stability of these principles and choices in the context of Covid-19.

In the design of our studies, the rationing policy questions always came after the choices. This could plausibly result in post hoc rationalization. However, we believe this potential bias does not undermine our findings; rather, it may lead to an underestimation of inconsistencies, which further supports a central point of our paper, that decisions are, if anything, even more inconsistent with preferred policies than our results show. Additionally, reversing the order of the questions would not necessarily eliminate this issue. Participants might then deliberately or unconsciously tend to align their choices with their previously stated policy preferences, thus creating a different but comparable form of consistency bias.

One may, however, object that treating a patient choice that deviates from the favored policy of a participants as an instance of inconsistency might be too harsh. In other words, choosing a policy as as the most favorite policy doesn’t mean not valuing other policies at all. In such a case, one might argue that if a participant’s most favored policy says they should choose a certain patient (A), but all the other arguments/policies she values point towards choosing another patient (B), then it might seem unfair to say the participant was inconsistent in choosing patient B. While we understand this concern, we believe it is essential to clarify the role and definition of a policy within the context of our studies. By definition, a policy is intended to provide a consistent rule to be applied across all cases. As such, selecting a policy as the most important implies a commitment to follow it universally, rather than combining it with other considerations on a case-by-case basis. To account for participants who do not wish to adhere strictly to a single policy, we provided the “Other” option, allowing them to describe a more nuanced approach. In both experiments, a very small percentage of participants chose the “Other” option (2% of laypeople and 9% of clinicians). Moreover, in the policy question, we told participants to assume the policy they choose would be used in a situation like the one that had been described to them earlier in the patient choice question. This suggests that there is little reason for participants to believe that the scenario described in the patient choice question was unusual compared to the general use cases of their favorite policies. Thus, we believe it’s reasonable to expect a “consistent” or rational participant to use their favorite policy to allocate the treatment to the patients presented to them.

The influence of irrelevant alternatives in allocating treatments to patients, as reflected in the attraction effect, combined with the inconsistency of such decisions with the desired policies of our participants, suggest that rationing decisions may be difficult for people (including clinicians) to make on a case by case basis. This is supported by informal examination of participants’ free response comments, which included several reports of feeling anxiety and other negative emotions while making choices in our hypothetical and quite abstract scenarios. For example:

“It was a little bit depressing having to consider these things ...”

“Stressful.”

“I was uncomfortable with playing God.”

“It was disturbing to think about it.”

Being required to apply a pre-established, generally accepted policy in such situations, rather than make decisions involving difficult tradeoffs on the spot, might be expected to reduce such negative emotions.

As a final note, it should be noted that a large fraction of our participants in the clinician experiment were nurses. However, nurses are often integral members of multidisciplinary healthcare teams involved in the development and implementation of crisis standards of care, including COVID-19 rationing policies. For example, [37] and [38] discuss the participation of nurses in these teams, emphasizing their contributions to triage decisions and the ethical frameworks guiding resource allocation. Furthermore, some published guidelines explicitly highlight the need for multidisciplinary teams that include nurse leaders in making these decisions, while others assign this responsibility to senior physicians [5]. Even in cases where nurses are not directly involved in creating guidelines, they frequently manage life-sustaining technologies and play a critical role in implementing triage decisions, including ventilator withdrawal and reallocation [5]. Beyond their involvement in explicit rationing decisions, nurses engage in implicit bedside rationing of their care, which is often referred to as “missed care” or “unfinished care.” This type of rationing involves allocating their time and attention across multiple patients and prioritizing care tasks, often without the guidance of formal frameworks. Evidence from systematic reviews [39] and recent studies, e.g., [40] and [41], indicates that such rationing can have significant consequences for patient safety and quality of care. These findings underscore the relevance of nurses to real-world rationing scenarios, even if their decisions are more implicit than those of physicians managing ventilators or medication distribution.

Although Covid-19 crisis standards of care are not currently active, future developments in Covid-19, such as vaccination and treatment-resistant variants, as well as future pandemics, such as a pandemic influenza, could make them necessary once again. Moreover, health systems that allocate care on the basis of need rather than ability to pay engage in more or less constant rationing [42] and a growing literature similarly documents that implicit rationing routinely, if less visibly, occurs when human resources, such as nurses, are in short supply (e.g., [43]). Good decision-making regarding allocation of scarce health resources therefore remains necessary. Our findings highlight the necessity of designing and communicating treatment allocation policies and implementation of practices that take into account how guidelines are applied (or fail to be applied) by frontline clinicians.

5 Results after exclusions

Table 7 presents the choice shares of different patients as well as the sample size in each condition for the laypeople and clincian experiments after excluding the participants who failed the attention/robustness check question (i.e., chose a dominated patient in one of those questions). After these exclusions the attraction effect grew a bit larger in both domains (Ventilator: Patient B in $[eqn]$ , Patient B in $[eqn]$ and Remdesivir: Patient B in $[eqn]$ , Patient B in $[eqn]$ ). We continued to fail to observe the compromise effect, although the negative compromise effect shrank (Ventilator: $[eqn]$ , Remdesivir: $[eqn]$ ). With both domains combined, we continued to observe a significant attraction effect (Patient B in $[eqn]$ , Patient B in $[eqn]$ ) but no compromise effect (χ2=0.15,df =1,p=.70). In the experiment with clinicians, the attractions effect fell by about 1% but remained significant ( $[eqn]$ ).

Table 7: Choice shares of patients in different conditions and experiments conditional on passing the attention checks. Each label in the Condition column refers to the labels of the patients that were presented in that condition. For example, in condition AB, patients A and B were presented

6 Inconsistency analysis after exclusions

Laypeople experiment. In the experiment with laypeople, after excluding the 201 people who were in the bottom 10th percentile, we ended up with 685 participants who had chosen one of the three relevant policies, 298 of which (44%) were inconsistent. After also excluding the inattentive participants, 653 of the remaining participants had chosen one of the three relevant policies, out of which 283 (43%) were inconsistent.

Clinicians experiment. In the experiment with clinicians, after excluding the 128 people who were in the bottom 10th percentile, we ended up with 437 participants who had chosen one of the three relevant policies, 193 of which (44%) were inconsistent. After also excluding the inattentive participants, 431 of the remaining participants had chosen one of the three relevant policies, out of which 190 (44%) were inconsistent.

7 Demographic analysis of the laypeople experiment

Tables 8 and 9 below presents the regression results on the relationship between the approval ratings of policies and the demographics of participants in the experiment with laypeople. Note that participants who have chosen “Prefer not to answer” in any of the demographic questions or have chosen “Other” in the sex, race, or the religion question have been excluded in this analysis (198 people were excluded). Also, people who have chosen multiple races are coded as “Mixed” with the categories being Asian, Black, Hispanic, and White, and people who have chosen a religion are coded as Religious (Religious=1) and non-religious, otherwise (Religious=0). Below are the definition of the other variables in the regression:

Table 8: Results of regressing the approval ratings of policies on the demographic variables, part 1.

Table 9: Results of regressing the approval ratings of policies on the demographic variables, part 2.

Combined pol=Political attitude index (a la. [44]). This is calculated by averaging the z-scores of the answers to the political ideology and political party questions. Political ideology is measured on a 5-point scale ranging from Very Liberal (1), Liberal (2), Neutral (3), Conservative (4), Very Conservative (5). Political Party is measured on a 7-point scale ranging from Very Democrat (1), Democrat (2), Somewhat Democrat (3), Neutral (4), Somewhat Conservative (5), Conservative (6), Very Conservative (7).

CAS_score: Covid anxiety scale. This is the z-score of the score on the Covid anxiety scale [26].

Education years: Less than High School=10, High School Degree=12, Some College=14, College=16 Some Graduate=18, Graduate Degree=20

Income value: Less than 20k=10, Between 20k and 40k=30, Between 40k and 60k=50, Between 60k and 80k=70, Between 80k and 100k=90, More than 100k=110

Tables 10 and 11 below presents the logistic regression results on the relationship between the opinions on whether or not a piece of information should be considered and the demographics of participants in the laypeople experiment. Table 12 presents policy preferences of laypeople across the two experimental conditions; ventilator and remdesivir. Finally, the appendix ends with the survey materials, first for the laypeople experiment and then, for the experiment with clinicians.

Table 10: Results of logistic regression of if different pieces of information must be used in a rationing policy on the demographic variables, part 1.

Table 11: Results of logistic regression of if different pieces of information must be used in a rationing policy on the demographic variables, part 2.

Table 12: Policy preferences of laypeople across the two experimental conditions; ventilator and remdesivir.