A Value-based Trust Assessment Model for Multi-agent Systems
Kinzang Chhogyal, Abhaya Nayak, Aditya Ghose, Hoa Khanh Dam

TL;DR
This paper proposes a value-based trust assessment model for multi-agent systems that considers shared values and trust behaviors, addressing trust evaluation in unfamiliar situations where past observations are limited.
Contribution
It introduces a novel approach to trust assessment based on shared values, incorporating trust cautiousness and dependence, filling gaps in traditional observation-based methods.
Findings
The model effectively evaluates trust using shared values.
It accounts for cautious and bold trust behaviors.
The approach improves trust assessment in new or unfamiliar scenarios.
Abstract
An agent's assessment of its trust in another agent is commonly taken to be a measure of the reliability/predictability of the latter's actions. It is based on the trustor's past observations of the behaviour of the trustee and requires no knowledge of the inner-workings of the trustee. However, in situations that are new or unfamiliar, past observations are of little help in assessing trust. In such cases, knowledge about the trustee can help. A particular type of knowledge is that of values - things that are important to the trustor and the trustee. In this paper, based on the premise that the more values two agents share, the more they should trust one another, we propose a simple approach to trust assessment between agents based on values, taking into account if agents trust cautiously or boldly, and if they depend on others in carrying out a task.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
A Value-based Trust Assessment Model for Multi-agent Systems
Kinzang Chhogyal1
Abhaya Nayak1
Aditya Ghose2
Hoa K. Dam2 1Macquarie University, Sydney, Australia
2University of Wollongong, Wollongong, Australia {kin.chhogyal, abhaya.nayak}@mq.edu.au, {aditya, hoa}@uow.edu.au
Abstract
An agent’s assessment of its trust in another agent is commonly taken to be a measure of the reliability/predictability of the latter’s actions. It is based on the trustor’s past observations of the behaviour of the trustee and requires no knowledge of the inner-workings of the trustee. However, in situations that are new or unfamiliar, past observations are of little help in assessing trust. In such cases, knowledge about the trustee can help. A particular type of knowledge is that of values - things that are important to the trustor and the trustee. In this paper, based on the premise that the more values two agents share, the more they should trust one another, we propose a simple approach to trust assessment between agents based on values, taking into account if agents trust cautiously or boldly, and if they depend on others in carrying out a task.
1 Introduction
Though vastly outnumbered and facing certain defeat in Thermopylae, Leonidas still trusted that his soldiers would stand and fight for Sparta with their lives. What made him have such faith in them? It is plausible that his prior experience of sharing the battlefield made him trust them. However, a more compelling reason and one that is of interest to us, could be because they shared common values: they valued their way of life, they valued courage, they valued their freedom and they valued Sparta.
Autonomous systems such as self-driving cars are becoming a common sight and they have become a source of trepidation in humans. It appears inevitable that we must coexist with them and such fears may be alleviated by designing systems that humans can trust. In computation, there are different perspectives from which to approach trust. An interesting perspective that has largely motivated this work is offered in Roff and Danks (2018) where two dimensions of trust are presented: one that depends on reliability and/or predictability and another that depends on one’s understanding of other people’s values, preferences, expectations, constraints, and beliefs, where that understanding is associated with predictability but is importantly different from it. It is this latter dimension which relies on the knowledge of others.
Many definitions of trust can be found in the literature. We adopt the following definition from Lee and See (2004): the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability. It is important to note that trust arises in situations where i) a trustor expects the trustee to perform some action, and that ii) trustors, in general, have no certainty about the motives and actions of the trustees. For a survey of trust models, see Sabater and Sierra (2005).
Out of the ‘reliability and/or predictability’ dimension and the ‘knowledge dimension’, the focus in AI has largely been on the former. For instance, one of the earliest works in computational trust Marsh (1994) was based on this dimension. The trustor in such cases relies on past observations of the trustee’s behaviour and has no deep knowledge of the trustee. For example, I trust my car will start in the morning without knowing the inner-workings of the car Roff and Danks (2018). The problem with this dimension is that since it relies on past experiences, if situations arise that are either new or unfamiliar, it is not clear how much to trust or even worse how to trust. This is especially important for autonomous agents as they may find themselves in worlds that are chaotic and ever-changing. They are certain to encounter situations that they have not seen before and choosing how much to trust another agent based on past experiences is futile. This is where trust based on the second dimension can help. The agent’s trust in another agent is a function of its knowledge of the latter. Such knowledge could consist of many things but an important factor in the context of trust is knowing what things are important to others, i.e. their values. For instance, if both you and your architect value beauty, you can trust your architect to deliver a design that is beautiful.
This paper is premised upon why Leonidas trusted his soldiers and why you could trust your architect – the sharing of common values. It is reasonable to assume that the more you share values with someone, the more likely you will trust them. We focus on agents that have to rely on other agents to execute certain actions for them but in order to do so they must find the most trustworthy ones. That is, they will seek agents that share their values. We begin by presenting a trust assessment model that relies on both the dimensions – reliability and value sharing. We then constrain our model to one where only values are used, as that is the focus of this paper. We briefly discuss what values are and how they may be used in trust assessment. Several different ways that trust may be assessed are presented. We end by discussing the limitations of this work and how it may be further extended.
2 A Trust Assessment Model
The scenario that we consider in this paper is an environment consisting of autonomous agents that can execute actions. Our work is motivated by the Belief-Desire-Intention (BDI) agent model Rao (1995) but we limit our discussion only to the features of BDI agents that are relevant to our work. Let represent the set of all agents. There is also a set which represents the set of all possible actions. Note that agents may not be able to execute every action in but they may still be aware of those actions and of other agents that can execute them. The goal of an agent may either be to change the state of the world or get some information about its current state.
Definition 1
Let be an agent with some goal and be another agent that can help achieve ’s goal by executing action . We define ’s trust assessment of w.r.t as:
[TABLE]
where and are weights, represents ’s trust assessment of based on reliability and predictability, and represents ’s trust assessment of based on its knowledge of .
If we take the measure of trust to be the probability with which thinks can help achieve its goal by executing , then . Since relies on past observations, it is implicit that has a history of executed actions to draw on that involve and this makes it amenable to machine learning techniques. However, it could turn out that no such history is available; in that case, can be taken to be [math] and therefore, . This will be the extent of our discussion of . We now turn to which is the main focus of the paper. The weight is not important and we ignore it in our discussion. In the rest of this paper, we will focus on only one kind of knowledge of the trustee, namely, its values. We refer to as ’s value-based trust assessment of w.r.t. action or simply trust assessment when it is clear from the context.
2.1 Values
Values are things that are important to us. According to Schwartz’s Theory of Basic Values Schwartz (2012), all values exhibit six features that include: i) being able to be activated and causing emotions to rise, ii) acting as goals that can motivate action, iii) guiding the selection of actions and, iv) being able to be ordered by importance. Additionally, in Schwartz (2012), ten broad values such as benevolence, power, security and conformity are identified under which more concrete values may fall.
Values may also be compatible with each other (conformity and security) or be in conflict with each other (benevolence and power).111Note the same pair of values might conflict in one context and not in another - so they may be context-sensitive. However, we do not take up context-sensitivity in this paper. Although one could argue that trust (trustworthiness) is itself a value, the central premise of this paper is that trust between two agents arises based on the compatibility of their values. This view of trust is in line with value sensitive design Friedman et al. (2013) which takes into account human values during the design process of systems which in our case is a trust assessment system.
We assume all agents have values that are explicitly programmed. The ten broad values mentioned earlier are useful but too coarse for our purpose. Those values are likely to be universal Schwartz (2012), meaning, they are likely present in all agents and differentiating agents based on those values is almost impossible. The values that we consider are therefore taken to be more concrete values which may be classified under these broader values. Agents may share values but they may also have personal values unique to them. Agents may have conflicting values but as in Schwartz (2012) we take that conflicting values are not pursued in a single action. This has an important implication that specific to each action is a set of non-conflicting values that the agent considers important.
Values are assumed to be activated when the state of the world changes due to an agent’s own actions or actions of other agents. As in Cranefield et al. (2017), we assume that for each value of an agent, there is a value state that represents the current level of satisfaction for the value. Value states could be affected both by an agent’s own actions or by the actions of other agents. For instance, an agent that donates money would increase the value state of generosity for itself. On the other hand, if the agent values the environment, the value state would decrease for this agent even if it is another agent that pollutes the environment. Furthermore, in Cranefield et al. (2017), value states are taken to be numbers that do not exceed a certain value. They are also assumed to decay to represent the fact that if no action has been taken in a while that advances an agent’s value, its satisfaction decreases. Our concern here is not so much about the actual values but more about the fact that value states can either increase or decrease. Given a set of actions and a set of values, we consider the agent’s choice of an action to be guided by the values. More specifically, an agent’s choice is such that: i) it increases the value state of each of its values and/or, ii) it minimises the number of values whose value state is decreased. The first condition is desirable but is not always achievable. For instance, you respect traffic rules but might run a red light in case there is a person requiring immediate hospitalisation. In this case, the value state for helpfulness would increase whereas the value state for law abidance would decrease. However, in this paper, we will assume an agent’s action increases the value state of each of its values related to that action. This is a strong assumption and will be addressed in the discussion section.
We now formalise the notions that were just discussed. We assume there is a set of all values, , from which an agent’s values are drawn. We also assume that it is possible for a value to have one or more opposing (conflicting) values in . The term is the set of opposing values of . However, if and , we abuse notation and write and also let stand for any opposing value of .
Definition 2
Let . We say is consistent iff for each . Otherwise, it is inconsistent.
Definition 3
Given two sets of values and respectively, the conflict set is defined as .
Ex 1
If , and , then and .
Ex.1 shows is not symmetric. Some basic properties follow from these definitions:
Proposition 1
Given two sets of values if one of or is consistent, then is consistent.
Note that even if and are both inconsistent, could be consistent. For instance, if where , and where , then which is consistent. On the other hand, even though both and are consistent, it can be that inconsistent. For instance, if , , where , then is inconsistent.
Proposition 2
Given two sets of values , if one of or is consistent, then is consistent.
Proposition 3
Given two sets of values , is inconsistent iff both and are individually inconsistent and there is some value such that both in and .
Proposition 4
Given three sets of values :
, 2. 2.
.
Proposition 4 shows that distributes over and . However, the converse doesn’t hold, i.e., and do not distribute over . We show them below along with the non-associativity of for the sake of completeness. For the counterexamples below, let , , and where .
:
Ex. We get , and . 2. 2.
:
Ex. We get , and . 3. 3.
.
Ex. We get , and .
2.2 Value-based Trust Assessment
Definition 4
An agent ’s value set, , is a subset of .
Definition 5
Given an agent and an action , the action value set associated with , denoted as , is a subset of that is consistent.
When it is clear from the context, we write simply as . Def. 5 follows from what we mentioned earlier that conflicting values cannot be pursued in a single action. We don’t specify how is formed but the values in it should consist of values that are important w.r.t . For example, if I am about to buy a new piece of furniture, I might care about functionality and not beauty; so functionality would be in . Note that we did not mention whether can be executed by or not. might not be able to execute an action but it can still be aware of the action and the values that are important relative to it. For instance, you may not know how to drive but in asking someone to drive, you would still value safety and comfort. The action value set could also consist of core values that are important to the agent regardless of any action. As mentioned earlier, if can execute , it is assumed that all values in increase their value state after executing .
Basic Trust Assessment
The first case we consider is how an agent might assess its trust in another agent when requesting a particular action to be executed.
Definition 6** (Two Agent - Independent)**
Given an action , two agents and with value sets and , the value-based trust assessment of by is defined as:
[TABLE]
Intuitively, the level of trust places in is determined both by the values they share, , and the extent to which ’s values conflict with ’s, . Note that . Also, is consistent from Proposition 2. We will at times annotate and write it as since is not acting on behalf of any agent. This is mainly to make the presentation simpler when comparing different trust assessment functions. The following properties result directly from Def. 6 :
if , , 2. 2.
if = {}, , and 3. 3.
if and , .
Ex 2
Let and , where and be some action. We get .
Next, we consider the case where three agents are involved. Say asks to build her a red chair. However, is only a carpenter and not a painter. So, must also request a trustworthy painter to paint the chair. We have to be careful here as there are two value sets concerning : and . The question is which value set does use in order to pick a painter ? Since is fulfilling ’s request, we assume that supersedes and is the value set used to choose , i.e. . If were acting independently of , then it would be more appropriate to take as . We propose two ways that might adopt to choose .
Definition 7** (Three Agents - Cautious)**
Given actions and , three agents , and where is executing on behalf of and is executing on behalf of , and value sets , and, , the cautious trust assessment of by is defined as:
[TABLE]
Here, we say trusts cautiously. It tries to pick an agent that has the most values common to both itself and . On the other hand, it avoids agents that have a lot of values in conflict with itself or . At times we use the annotated form . Note that the relevant action in is though is defined relative to , i.e. . may be inconsistent but since is consistent, from Proposition 2, we know is consistent.
Ex 3
As in the previous example, let and , where . Let where . .
Definition 8** (Three Agents - Bold)**
Given actions and , three agents , and where is executing on behalf of and is executing on behalf of , and value sets , and, , the bold trust assessment of by is defined as:
[TABLE]
Here, we say trusts boldly. The annotated form is . As in the previous case, values common to all three agents are considered but so are values that and independently share with for assessing the trust in . In general, places at least as much trust in agents as it would have when being cautious as shown in Proposition 5 below.
Ex 4
As before, and , where . Let where . .
Proposition 5
Given actions and , three agents , and with value sets , and where is executing on behalf of and is executing on behalf of , .
When trusts boldly or cautiously, it assesses its trust in for executing with ’s value set in mind. It is interesting to see what ’s trust in would be if it ignores . We say is acting semi-independently because we still take as and not . The definition for is the same as in Def. 6:
Definition 9** (Three Agents - Semi-Independent)**
Given actions and , three agents , and with value sets , and, , the trust assessment of by is defined as .
Ex 5
As before, , where and where . .
From Ex.3, Ex.4 and Ex.5, we see that is greater than or . In other words, trust that places in when acting semi-independently is greater than when it is acting on behalf of . However, this only holds in general between and , and is shown in the next proposition.
Proposition 6
Given actions and , three agents , and with value sets , and, , .
The following counterexample shows that is not true in general.
Ex 6
As before, let and , where . We change to where . . .
For the special case, when no two of , , have conflicting values with each other, we have the following result:
Proposition 7
Given actions and , three agents , and with value sets , and that have no conflicting values with each other, .
Trust Sequences
We now turn our attention to trust sequences when a series of agents are involved in assessing trust.
Ex 7
Consider agent has to achieve a goal that requires the execution of a particular action . , however, cannot execute and instead must rely on another agent. Assume is only aware of agents and that can execute .
In order to pick the best one amongst the two, chooses the one that it believes to be more trustworthy. It does this by assessing its trust in and , and respectively.
Ex 8
(cont.) Suppose has picked to execute the action as .
As seen in the example, uses a simple rule to pick or . There are two reasons for this: i) can maximise the chance of its value states increasing, by picking an agent with whom it shares the most number of values, and ii) by choosing agents with whom it has fewer conflicting values, it minimises the chance of its values being violated. The best scenario for is the case where either .
Ex 9
(cont.) Assume that , in turn, has to request either or to execute another action to fulfil ’s request.
Similar to what did, assesses its trust in and . Since three agents will be involved , and, or , we use either Def. 7 or Def. 8. Similar to the case for two agents, picks the greater of and .
Ex 10
(cont.) Assume chooses using Def. 7 who then executes which is the last action to be executed. The trust assessments between , and , where and are the chosen agents form a trust assessment sequence as shown:
[TABLE]
We now formally define a trust assessment sequence.
Definition 10
A value-based trust sequence or simply a trust sequence is a sequence of trust assessments, , where , represents agent ’s trust assessment of agent w.r.t to action and .
Shown below is a way to visualise a trust sequence. Trust assessments on either side are surrounded by the agents involved.
[TABLE]
The trust sequence above is initiated by (initiator) and is the initial assessment. All other assessments will be referred to as subsequent assessments. The last agent in the sequence to execute an action is and is called the terminator. For all , each represents the agent that was chosen to execute action by agent . The length of the sequence is equal to the number of trust assessments, i.e. above. The condition prevents sequences where agents assess trust in themselves.222It may be possible that an agent appears again in some other place in the sequence. The number of agents involved in the sequence is therefore at most . In this paper, we only consider sequences where at each step, an agent only has one trustee. For instance, in Ex.10, there are no other agents besides that asks to execute an action and similarly there is only for . This leads a sequence that has no branches. Ex.10 already showed how trust sequences are generated and now we present it more formally.
Definition 11
Let , be an agent looking for another agent to execute action . The value set of is . For each where , that can help execute , we define:
[TABLE]
where if , is given by Def. 6 and if , is given by one of Def. 7 or Def. 8.
It is clear all trust sequences use Def. 6 but differ on whether they use Def. 7 or Def. 8. This point forward by a cautious trust sequence we mean one that uses Def. 7 and by a bold trust sequence we mean one that use Def. 8 for all .
Definition 12
Given a trust sequence of length , the aggregate trust of the trust sequence is equal to and is denoted as .
During each trust assessment step in the sequence, we are computing the difference between the number of values that are shared and the number of values that are in conflict; is simply the sum of those differences. If it is positive, then as a whole there are more values preserved between each step of the sequence compared to the number of values that are in conflict; if it is negative, the converse is true. Def. 12 also allows us to compute the aggregate trust of a subsequence: , where and .
In Def. 11, may trust either boldly or cautiously to choose an agent . An interesting question to ask is whether being bold or cautious makes any difference at all, i.e. will always select the same agent irrespective of whether it is trusting boldly or cautiously? As the example below shows, being cautious or bold matters.
Ex 11
Given actions and and four agents , , and where is executing on behalf of and has to choose one between and for executing , let , , and . Consider first: . Similarly, . will choose if trusting cautiously. Consider now: . . So, will choose if trusting boldly which if different from the previous case.
Intuitively, we think of as representing ’s trust assessment of w.r.t . What is not clear is whether should be updated to ? The reason for this is because ’s trust in also depends on whether has chosen a trustworthy agent that can help fulfil ’s goal. Assuming we do so, the implication of Theorem 1 below is that if is used to update ’s trust in , then the updated value of ’s trust in will be greater if agents in the sequence trust boldly and not cautiously.
Theorem 1
The aggregate trust of the trust sequence resulting from is greater than or equal to the aggregate trust of the trust sequence resulting from , i.e. .
3 Discussion
We discuss some limitations of our work and how it may be expanded on in the future.
Bias in bold agents** ** Consider again Def. 8 of a bold agent:
[TABLE]
Say has selected as is the maximum. For simplicity, assume there are no conflicting values in , and . We know . Assume that is much bigger than . Observe that is largely biased towards compared to as they share more values. This means in future trust assessments starting with , ’s values could be ignored as more of ’s values carry over to the next step in the sequence compared to ’s. Now if there happened to be another agent such that is only slightly smaller than but is only slightly bigger than , it seems might be a better choice than because as many of ’s values are as likely to be preserved as ’s. This leads to the slightly more complex definition for bold agents below:
[TABLE]
A similar kind of bias might exist in the subtrahend of Def. 8, i.e. between and . However, we think minimising the total number of conflicting values heavily outweighs the importance of minimising the bias in this case, so accounting for it is probably unnecessary.
Aggregate trust of a sequence and trust update We mentioned previously the possibility of updating to or some other value that is a function of it. The case where seems plausible as we can reason that may have overestimated its trust in because it had no knowledge of other agents involved. However, if , explaining why ’s trust in should increase is not easy. This suggests that as a basis of trust update might have to be applied in a more sophisticated way.
Value Preservation Given a trust sequence of length , it would be convenient to have a measure which at a minimum could tell us whether a value in the initiator is also in the terminator without having to inspect the values of all agents involved. The aggregate trust of the sequence, , doesn’t seem to have the right characteristics for this. A multiplicative measure based on the ratio between the number of values preserved and the number of values in conflict for each trust assessment is one possible option to explore.
Value Preferences We did not consider preferences over values such as in Serramia et al. (2018). Suppose you have to choose between two hotels, one in the Downtown area close to all the local attractions and the other cheaper but requiring more travel. If you value convenience more than price, then you would choose the Downtown hotel whereas if you value price more, you would book the cheaper one. When another agent is involved, you will likely choose an agent that has preferences over values similar to yours. This requires more knowledge and also brings additional complexity. A possible way of doing this is to modify the trust assessment functions in Def. 6, Def. 7 and Def. 8 so that they use a measure such as Kendall’s tau distance Kendall (1938).
Value States Although we mentioned that values can be activated and their value states can either increase or decrease, we did not consider it in our model. Incorporating this information into will be an interesting way to build on the model. We briefly discuss one way this might be done. Let and be two agents with value sets and and be an action that is executing on ’s behalf. Let and be the set of values in whose value state increases and decreases due to the execution of respectively. Then:
[TABLE]
are values shared by and whose value state increases and, are values share by and whose value state decreases. We could then rewrite the trust assessment function in Def. 6 for two agents as:
[TABLE]
where and are weighting factors. Note that in Def. 6 has been replaced by . Both and in Def. 6 they both contribute positively. We subtract them but we want to be careful that they don’t equal to zero if and thus the use of weighting factors. Values in could also increase and decrease but since they are all in conflict with , we do not differentiate between such values. Similar functions for both Def. 7 and Def. 8 can be constructed.
Public Values and Action Decomposition We assumed that when agent is assessing its trust in agent , the values of are publicly visible to , i.e. is certain of ’s values. This is quite a strong assumption. A way to circumvent this is to instead consider the set of values that believes has. Also, in an earlier example, we considered the task to build a red chair and we alluded to the fact that there were two actions involved: build and paint. More work is required on this aspect of decomposing complex actions into simpler ones. 333We are thankful to a anonymous for pointing these issues out and for suggesting that instead of knowing for certain, agents could perhaps hold beliefs of what another agent’s values are.
4 Conclusion
We presented a simple approach to how values can be used by agents to assess their trust in each other. We defined the notion of value-based trust assessment functions and showed how they lead to trust sequences. Many of the ideas in this paper could be further expanded upon and explored in more detail, and there is much to uncover about how values and trust are related. We leave it to our future research.
Acknowledgement
We would also like to thank anonymous referees for their comments.
5 Appendix
See 1
Proof 1
1. Follows from the fact that to make inconsistent it must be that there is a such that both and are in and . But at least one is consistent, so it can’t be that inconsistent.
See 2
Proof 2
There are three cases to consider. (1) consistent, consistent: Assume is inconsistent. This means there is a value such that both and from Def. 3 this implies that both . However, is consistent and we get a contradiction. (2) consistent, inconsistent: Proof similar to case 1. (3) inconsistent, consistent: Assume is inconsistent. Again for some , both . From Def 3, it must be that and , and and . However, as is consistent, it cannot have both and which gives us a contradiction.
See 3
Proof 3
Left to Right*: Assume is inconsistent. Then from Prop. 2, both must and are inconsistent. Assume for contradiction, no value such that both in and . By Def 3, for any , at most one of or in as cannot contain both and . This means obtained is consistent and leads to a contradiction. Right to Left: Assume in and . By Def 3, both in which makes it inconsistent. *
See 4
Proof 4
*1. : We show that if some then it must also be in and vice versa. Left-hand Side: Let some . Then it must be that and . Since and , it must be that and . Thus, . Right-hand Side: Let some . Then and or , and . Thus and therefore
- : We show that if some then it must also be in . Left-hand Side: Let some . Then and . There are three cases to consider. a) : Thus and therefore, . b) : Similar to previous case. c) : Similar to previous case. Right-hand Side: Let . There are three cases to consider. a) : Then which means and therefore . b) : Similar to previous case. c) : Similar to previous case. *
See 5
Proof 5
The minuend in and the minuend in . Since , it follows . The subtrahends are the same, so it must be that .
See 6
Proof 6
We know . We know . Since , it follows . Also, we know from Prop. 4 that , so it follows that . Thus, .
See 7
Proof 7
We already know from Prop.5 and Prop.6 that is less that or equal to and . Since there are no conflicts of values between agents and . , so and thus .
See 1
Proof 8
Take any sequence of length constructed using . It is enough to show that we can construct a sequence using whose aggregate trust is greater or equal to that of . For , since we must use for both and , . When and , for , let and let be some agent . Now for , if there is an agent such that , then as we previously established . If there isn’t one, for , we can still choose and we know from Prop. 5 that , thus . We can reason similarly for and hence .
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Cranefield et al. [2017] Stephen Cranefield, Michael Winikoff, Virginia Dignum, and Frank Dignum. No pizza for you: value-based plan selection in bdi agents. In Proceedings of the 26th International Joint Conference on Artificial Intelligence , pages 178–184. AAAI Press, 2017.
- 2Friedman et al. [2013] Batya Friedman, Peter H Kahn, Alan Borning, and Alina Huldtgren. Value sensitive design and information systems. In Early engagement and new technologies: Opening up the laboratory , pages 55–95. Springer, 2013.
- 3Kendall [1938] Maurice G Kendall. A new measure of rank correlation. Biometrika , 30(1/2):81–93, 1938.
- 4Lee and See [2004] John D. Lee and Katrina A. See. Trust in automation: Designing for appropriate reliance. Human factors , 46 1:50–80, 2004.
- 5Marsh [1994] Stephen Paul Marsh. Formalising trust as a computational concept . University of Stirling, 1994.
- 6Rao [1995] AS Rao. BDI agents: From theory to practice. In Proc. of the First Intl. Conference on Multiagent Systems (ICMAS-95), San Francisco , pages 312–319, 1995.
- 7Roff and Danks [2018] Heather M Roff and David Danks. “Trust but Verify” : The difficulty of trusting autonomous weapons systems. Journal of Military Ethics , pages 1–19, 2018.
- 8Sabater and Sierra [2005] Jordi Sabater and Carles Sierra. Review on computational trust and reputation models. Artificial intelligence review , 24(1):33–60, 2005.
