Using Social Choice Theory to Finalize Architectural Decisions

TL;DR

This paper models architectural decision-making as a social choice problem, revealing vulnerabilities to stakeholder manipulation and proposing incentive-based mechanisms like VCG to ensure unbiased decisions.

Contribution

It introduces social choice theory and mechanism design to software architecture, demonstrating how to mitigate stakeholder bias using VCG mechanisms.

Findings

Most decision methods are vulnerable to strategic manipulation.

Incentive mechanisms can protect decision integrity.

VCG mechanism effectively reduces stakeholder bias.

Abstract

Unbiased and objective architectural design decisions are crucial for the success of a software development project. Stakeholder inputs play an important role in arriving at such design decisions. However, the stakeholders may act in a biased manner in order to ensure that their requirements and concerns are addressed in a specific way in the software. Most of the existing methods of architectural decision-making do not adequately account for such biased behaviour of stakeholders. We view the software architecture design as a collective decision making (CDM) problem in social choice theory, and introduce the central ideas of mechanism design to our field. Our contributions are twofold: i) Using the impossibility results from social choice theory, we show that a rational stakeholder can game, to her advantage, almost every known method of software architecture decision making. ii) We also show that if the architect is willing to bear the extra cost of giving suitable incentives to stakeholders, then architectural decision making can be protected from strategic (biased) manipulations. For achieving this objective, we introduce the Vickrey-Clarke-Groves (VCG) mechanism from social choice theory to the software architecture domain. We illustrate our contributions by comparing and using the examples drawn from the well-known CBAM method of architectural decision making.

Tables

Table 1. Table 1 : CBAM based on actual stakeholder benefits for architecture strategies

	Strategy AS1	Strategy AS2	Strategy AS3
Actual Benefits of stakeholder, $s_1$	80	70	65
Actual Benefits of stakeholder, $s_2$	-90	50	60
Actual Benefits of stakeholder, $s_3$	-50	50	62
Average Actual Benefit, B	-20	56.67	62.33
Anticipated Cost, C	80	95	90
$Desirability=B÷C$	-0.25	0.6	0.69

Table 2. Table 2 : CBAM based on reported stakeholder benefits for architecture strategies. Wherever the reported values are different from actual values, they have been shown in bold font.

	Strategy AS1	Strategy AS2	Strategy AS3
Reported Benefits of stakeholder, $s_1$	80	50	-10
Reported Benefits of stakeholder, $s_2$	-90	50	60
Reported Benefits of stakeholder, $s_3$	-50	50	62
Average Reported Benefit, B	-20	50	37.33
Anticipated Cost, C	80	95	90
$Desirability=B÷C$	-0.25	0.53	0.41

Table 3. Table 3 : Manipulating dictatorial CBAM where the benefits reported by s 2 subscript 𝑠 2 s_{2} are taken as average actual benefits

	Strategy AS1	Strategy AS2	Strategy AS3
Actual Benefits of stakeholder, $s_2$	50	80	-10
Reported Benefits of stakeholder, $s_2$	30	100	-10
Anticipated Cost, C	40	100	50
Actual Desirability $=$ Average Actual Benefit $÷$ C	1.25	0.8	-0.2
Calculated Desirability $=$ Average Reported Benefit $÷$ C	0.75	1	-0.2

Table 4. Table 4 : Computation of payments and net benefits as per 𝒱 ​ 𝒞 ​ 𝒢 𝒱 𝒞 𝒢 \mathcal{VCG} mechanism, based on reported benefits r 1 subscript 𝑟 1 r_{1} , r 2 subscript 𝑟 2 r_{2} , and r 3 subscript 𝑟 3 r_{3} .

$i$	$s_1$ ($i=1$)	$s_2$ ($i=2$)	$s_3$ ($i=3$)
$T_i^{+}(r_1,r_2,r_3)$	100	100	100
$T_i^{-}(r_1,r_2,r_3)$	122	100	100
$p_i^{𝒱𝒞𝒢}(r_1,r_2,r_3)$	-22	0	0
$u_i\left(𝒱𝒞𝒢(r_1,r_2,r_3)\right)$	50	50	50
$N{B}_i^{𝒱𝒞𝒢}(r_1,r_2,r_3)$	28	50	50

Table 5. Table 6 : Computation of payments and net benefits as per 𝒱 ​ 𝒞 ​ 𝒢 𝒱 𝒞 𝒢 \mathcal{VCG} mechanism, based on reported benefits u 1 subscript 𝑢 1 u_{1} , r 2 subscript 𝑟 2 r_{2} , and u 3 subscript 𝑢 3 u_{3} .

$i$	$s_1$ ($i=1$)	$s_2$ ($i=2$)	$s_3$ ($i=3$)
$T_i^{+}(u_1,r_2,u_3)$	150	110	160
$T_i^{-}(u_1,r_2,u_3)$	150	110	160
$p_i^{𝒱𝒞𝒢}(u_1,r_2,u_3)$	0	0	0
$u_i\left(𝒱𝒞𝒢(u_1,r_2,u_3)\right)$	60	80	50
$N{B}_i^{𝒱𝒞𝒢}(u_1,r_2,u_3)$	60	80	50

Equations

$$d_j=\frac{{\sum }_{i=1}^n{r}_i(ASj)}{n\cdot c_j}$$

$$u_i(\mathcal{A}(r_1,r_2,\dots ,u_i,\dots ,r_n))\ge u_i(\mathcal{A}(r_1,r_2,\dots ,r_i,\dots ,r_n))$$

$$T_i^{+}(r_1,r_2,\dots ,r_n)=\sum _{j\ne i}{r}_j(a_{k^{\ast }})$$

$$T^{-}_{i}(r_{1},r_{2},\ldots,r_{n})=max_{k\in[1,m]}\Big{(}\sum_{j\neq i}r_{j}(a_{k})\Big{)}$$

Full text

We view the software architecture design as a collective decision making (CDM) problem in social choice theory, and introduce the central ideas of mechanism design to our field. Our contributions are twofold: i) Using the impossibility results from social choice theory, we show that a rational stakeholder can game, to her advantage, almost every known method of software architecture decision making. ii) We also show that if the architect is willing to bear the an extra cost of giving suitable incentives to stakeholders, then architectural decision making can be protected from strategic (biased) manipulations. For achieving this objective, we introduce the Vickrey-Clarke-Groves (VCG) mechanism from social choice theory to the software architecture domain.

We illustrate our contributions by comparing and using the examples drawn from the well-known CBAM method of architectural decision making.

1 Introduction

“Architectures are created solely to meet stakeholder needs” - [14]

Designing architecture is one of the most decisive phases of the software development life cycle. The architecture of a software relates the goals of stakeholders to different aspects of the requirements and then the well-defined requirements to the actual code of the project. This ensures traceability between the software’s motivation, objectives, and the final product. The traceability is enabled via software architecture and its related artifacts.

Architectural design decisions describe the rationale that led to the development of software architecture. Such design decisions were earlier considered as one of the architectural artifacts[17]. However, in recent years a paradigm shift in project development methodologies has made researchers and practitioners to consider the architectural design decisions as software architecture itself [18].

Every architectural decision, and in turn the entire software architecture, should be able to address the validated expectations of the stakeholders and provide a practical solution for it. Therefore, arriving at the architectural decisions is often considered difficult and is fraught with human biases. The human bias can originate from the architect or from stakeholders or both [18] [16]. Also, the existing techniques for arriving at software architecture decisions do not have a formal mechanism to handle stakeholders input bias [8]. To avoid such human bias and to choose an architecture that is fair to all stakeholders we propose a fair model of selecting an architecture.

Various frameworks such as CBAM [4] (which is based on ATAM [6]), [7], etc. have been developed for choosing the best candidate architecture from the perspective of modern economic theory. However, a significant assumption taken by these frameworks is the truthfulness of stakeholder inputs, even when it is against the stakeholder’s personal benefit. This is in stark contrast to the well-accepted assumption in modern microeconomics that all agents participating in any economic activity are selfish and rational, and are only concerned with their personal benefits [13]. Although CBAM-2 [11] highlights the stakeholder’s tendency to manipulate the architectural decisions, it fails to offer a formal mechanism to handle human bias. CBAM-2 merely provides a generic discussion-based approach to handle such situations and they suggest to altogether replace the voting based decision making.

Therefore, we address the following two questions in this paper:

Question-1. How do the “economic frameworks” for choosing a “most suited” software architecture perform when faced with the possibility of selfish stakeholders, who have the natural objective of influencing the decision processes to their (personal or group) advantage?

Question-2. How can an organization set up incentives to ensure truthful reporting and prevent collusion by various stakeholders while designing software architecture?

To the best of our knowledge, the above questions have not been studied adequately in the software architecture decision making literature. There exists no mechanism which ensures that the stakeholders reveal their true concerns and priorities, especially in relation or response to the other stakeholders’ inputs. Using the tools of game theory and mechanism design developed in the field of microeconomics, in this paper, we initiate the first systematic and detailed study of these questions.

The rest of the paper is presented as follows. In §2 we present our objective by explaining the concept of stakeholder rationality assumption and prove it with the help of a well-known example. §3 explains the social choice theory in the context of architectural design decisions. In order to address the Question-1 that we highlighted in the preceding paragraph, in §4 we give a rigorous proof of our claim that all of the architectural design decisions are manipulable. Further, to address the Question-2, we present in §5 a non-manipulable algorithm that ensures unbiased decisions during the architecture design. §6 discusses other economic models and frameworks in software architecture domains, and finally, §7 concludes our work and explains our future work.

2 Motivation and Objective

Different stakeholders in a software development project usually have different concerns or goals. However, such concerns may overlap each other, or be confirming to each other, or even may conflict. Typically, the benefit of a particular architectural decision to a stakeholder is expected to be aligned with the business, technological or quality goals championed by that stakeholder.

Current methodologies for software architecture design inherently assume that every stakeholder supplies to the architect an accurate picture of their inputs and preferences among the various competing architectural designs. In practice, this assumption is usually untenable [10] as it stipulates that each stakeholder will be truthful about one’s inputs, even if it leads to their personal loss. In reality, stakeholders are rational “players” and they are capable of influencing the architectural decisions for their own benefits. Further, these “players”, instead of manipulating the game alone, may also act in unison to have an even greater effect on the architectural decisions. We show how such tactics may work in §2.2 by considering the example of a well-known architecture evaluation method. Before that, we explain the rationality assumption.

2.1 The stakeholder rationality assumption

It is common knowledge and experience that each stakeholder acts rationally and in her own best interest. An important consequence of such rational (or, rather “selfish”) behavior of the stakeholder is that she may strategically report an inaccurate or suitably modified picture of her true preferences. She may do it to ensure that the overall architectural decisions are more favorable to her own goals.

An even more unfortunate consequence of stakeholder rationality may be collusion: a group of stakeholders decides together on reporting manipulated preferences so that the architect can be induced to select a particular software architecture design of their liking.

2.2 Effects of stakeholder rationality on CBAM method

We now illustrate the impact “stakeholder rationality assumption” can have on the dynamics and outcome of a given architecture selection procedure. For this purpose, we take an example with the well-known CBAM methodology [4] for most suited architecture selection.

CBAM in a nutshell (Adapted from [4])

Stakeholders report their benefits for every architecture strategy to the architect. We use $r_{1},r_{2},\ldots,r_{n}$ to denote the reported benefits. In particular, for each $1\leq i\leq n$, $r_{i}(ASj)$ denotes the benefit reported for architecture strategy $ASj$ by stakeholder $s_{i}$. The benefits are in the range $[-100,100]$.

Costs of various architecture strategies are estimated. Let $c_{j}$ be the mean cost estimated for architecture strategy $ASj$. Usually, there will also be an uncertainty interval around $c_{j}$.

Desirability scores for architecture strategies are computed. The desirability score $d_{j}$ of $ASj$ is equal to average reported benefit divided by mean cost:

$d_{j}=\frac{\sum_{i=1}^{n}r_{i}(ASj)}{n\cdot c_{j}}$

Architecture strategies with high desirability scores are compared based on uncertainty in their costs and benefits.

One or more architecture strategies with high desirability score and low uncertainty are selected.

Note: The above outline of the CBAM method is sufficient for the purposes of this paper. However, we would like to note that there is a detailed process attached to each of the above basic steps.

For ease of understanding, we have used the same terminology as used in CBAM for describing the following example. Suppose we have a scenario where the architect must choose an architecture for software such that certain quality attributes (QAs) are satisfied to the desired level. Further, suppose that a method such as CBAM is used to help in making such a selection. To keep the scenario simple, let’s say there is only one QA under consideration: battery lifetime, and assume that there are only three architecture strategies $AS1$, $AS2$, and $AS3$ affecting this QA, and three stakeholders $s_{1}$, $s_{2}$, and $s_{3}$ in the decision-making process. The architect has decided to employ CBAM methodology and will be selecting exactly one architecture strategy out of $AS1$, $AS2$, and $AS3$.

As per CBAM methodology, the architect first asks each stakeholder to report the contribution scores between $-1$ and $1$ for each candidate architecture strategy. The benefits of a candidate architecture strategy for a particular stakeholder is obtained by multiplying contribution score by $100$. Suppose the actual benefits derived are as given in Table-1.

The average benefit for each architecture strategy is also depicted in Table-1. The last two rows of this table contain the cost of each architecture strategy, as well as its desirability score as per CBAM methodology.

From the data shown in Table-1, one can, therefore, conclude that, if all stakeholders had reported their true and correct benefit values to the architect, $AS3$ would have been selected as per the CBAM method. This is because the strategy $AS3$ has the highest desirability. This can be pictorially seen from Fig-2, where cost is on $X$-axis, the benefit is on $Y$-axis, and cost-benefit evaluation of each stakeholder for each architecture strategy is represented by a point.

Now, let us change the scenario to selfish stakeholders, each working towards their own advantage. Suppose stakeholder $s_{1}$ somehow gets to know that her first preference $AS1$ will never get selected, as it is scored very low by the remaining stakeholders. Suppose the next best outcome for $s_{1}$ is the selection of $AS2$, since she prefers it over $AS3$. However, the CBAM method with truthful reporting will select her least preferred alternative $AS3$.

To tip the decision process in favour of $AS2$ over $AS3$, $s_{1}$ will strategize by falsely reporting a much higher benefit for $AS2$ and a much lower benefit for $AS3$. Suppose $s_{1}$ reports the benefit of $AS1$ at its actual value $30$, but falsely reports a higher benefit of $50$ for $AS2$ (we are assuming that $s_{1}$ is very lucky in choice of her false values) and a lower benefit of $-10$ for $AS3$. Further, suppose that the stakeholders $s_{2}$ and $s_{3}$ chose to give non-manipulated inputs. The outcome of CBAM method is now given in Table-2. This situation is pictorially represented in Fig-2.

Clearly, architecture strategy $AS2$ will be selected instead of $AS3$ now, as it has the maximum desirability score as well as due to the fact that the uncertainty in the desirability score is zero (all three stakeholders report the same value of $50$ for benefit from strategy $AS2$). Thus, by strategically misreporting her benefits based on available information about stakeholder choices, stakeholder $s_{1}$ was able to ensure $AS2$ gets selected instead of $AS3$. Since she prefers $AS2$ more than $AS3$, she has been able to influence the CBAM method in favor of her preferences.

To summarize, in the event of truthful reporting, $AS3$ would have been the option with maximum desirability score and minimum risk or uncertainty. However, strategically biased reporting by stakeholder $s_{1}$, leads to $AS2$ taking the place of $AS3$ as a preferred option in terms of costs, benefits, and uncertainty. We would like to note that the dynamics of CBAM method will be much more complicated when all stakeholders have access to various amounts of insider information about judgments of other stakeholders, and hence will misreport their benefits, by strategizing based on available information, either in groups or individually.

Given the existing gap in the architectural design decision-making process, we propose a decision-making method using social choice theory. Our approach can be used to find the optimal architecture in a given scenario. We propose an objective voting based mechanism which can avoid manipulation of the architecture selection process. We believe that a discussion-based method for the purpose is often very time consuming and subjective. Also, discussions are difficult to converge in complex scenarios.

3 Software architecture design in the context of social choice theory

The main thesis of this paper is that the software architecture design process can be naturally modeled as a collective decision making (CDM) problem in social choice theory. In a social choice theory framework, a CDM problem can be described as follows.

We have a set $S^{\prime}=\{p_{1},p_{2},\ldots,p_{n}\}$ of $n$ participants and a finite set $X^{\prime}=\{x_{1},x_{2},\ldots,x_{m}\}$ of $m$ alternatives or outcomes. Each participant $p_{i}$ has a particular preference order or valuation function $v_{i}:X^{\prime}\rightarrow\mathbb{R}$ over the set $X^{\prime}$ of all alternatives. The objective is to collate all individual preferences or valuations to arrive at a common preference or valuation over the set of all alternatives. Further, the choice of a common preference or valuation over the alternative set $X^{\prime}$ must be appropriate from the perspective of overall “social welfare”.

We now compare the above situation with a typical software architecture design process. An architect has to design a particular software architecture which rationally collates the viewpoints and preferences of various stakeholders involved with the software in varying capacities. Examples of stakeholders are acquirers, developers, maintainers, testers, end users, etc. [14].

3.1 CDM modeling

The mapping to social choice theory is as follows:

The set of possible software architectures (or architecture strategies) form the set $X=\{a_{1},a_{2},\ldots,a_{m}\}$ of alternatives. 2. 2.

The set of different stakeholders form the set $S=\{s_{1},s_{2},\ldots,s_{n}\}$ of participants. 3. 3.

Each stakeholder $s_{i}$ has some benefit (utility) $u_{i}(x)\in\mathbb{R}$ for every candidate architectural strategy $x\in X$. Alternatively, in case of preferences, each stakeholder $s_{i}$ has a linear ordering $o_{i}$ of the set $X$. If $x$ occurs before $x^{\prime}$ in ordering $o_{i}$, it means that stakeholder $s_{i}$ prefers architecture strategy $x^{\prime}$ over $x$. 4. 4.

The goal of the software architect is to choose an architecture strategy $x^{*}\in X$ which maximizes a suitable notion of social welfare i.e., an architecture which rationally collates the various preferences received by the architect from the stakeholders.

4 Impossibility of designing a non-manipulable software architecture selection method for selfish stakeholders

In §2.2, we illustrated by means of an example that the outcome of the well-known CBAM method for software architecture selection can be manipulated by selfish stakeholder(s) in their favor. In light of this example, we now consider the following natural question:

Is there a software architecture selection method which cannot be strategically manipulated by selfish stakeholder(s) in their favor?

In this section, we answer the above question in the negative i.e., we show that every stakeholder-input based software architecture selection method can be gamed. This has two consequences:

There does not exist any CBAM-like method which is non-manipulable, i.e., a method which is immune to stakeholder manipulation as described in 2.2. 2. 2.

The existence of the example discussed in §2.2 is not particular to the CBAM method; similar examples can be constructed for any other software architecture selection method which selects an optimal architecture based on stakeholders’ inputs.

The rest of this section is organized as follows. We start by providing some definitions and setting up relevant notation. In §4.1, we provide an abstract definition of a software architecture selection method, and we also give a rigorous definition of a non-manipulable software architecture selection method. (Such methods are called dominant strategy incentive-compatible in social choice theory literature [13][3][10].)

Finally, in §4.2, we show that no non-trivial non-manipulable software architecture selection method exists.

4.1 Definitions

Definition 1

Software architecture selection method. A software architecture selection method $\mathcal{M}$ with stakeholder inputs consists of the following sequence of steps:

The architect requests each stakeholder $s_{i}$ to provide her actual benefit $\mathbf{u_{i}}:X\rightarrow\mathbb{R}$. 2. 2.

Reported benefit by each stakeholder is $\mathbf{r_{i}}:X\rightarrow\mathbb{R}$. $r_{i}$ takes into consideration the fact that a selfish stakeholder may misreport the actual or true benefit. 3. 3.

By applying her decision algorithm $\mathcal{A}$ with the reported benefits $\{r_{i}~{}|~{}1\leq i\leq n\}$ as input, the architect selects a single architecture $a_{i^{*}}$ from the set $X$ of candidate architectures.

The key difference among the various software architecture selection methods lies in the design of the algorithm $\mathcal{A}$. For a CDM based on social choice theory, the algorithm $\mathcal{A}$ is called the social choice function [13, 3].

Definition 2

Non-manipulable software architecture selection. A software architecture selection method $\mathcal{M}$ is called non-manipulable if and only if it is in the best interest of each stakeholder to report her true benefit to the architect i.e., for all $1\leq i\leq n$ and for all possible reported benefit functions $r_{1},r_{2},\ldots,r_{n}$, we have that:

[TABLE]

Note that the above condition ensures truthfulness on the part of each rational stakeholder, irrespective of her awareness about the strategies as well as the actions of other stakeholders.

4.2 Non-existence of a non-manipulable software architecture selection method

The non-existence of non-manipulable software architecture selection methods follows from the Gibbard-Satterthwaite theorem [13][3] in social choice theory. For the scenario of software architecture selection, we adapt and state the theorem as follows:

Theorem 4.1

Suppose there are at least three different candidate architectures in set $X$. Further, suppose the social choice algorithm $\mathcal{A}$ used by the architect is non-manipulable.

Then $\mathcal{A}$ is a dictatorship, i.e., there exists an index $i$, $1\leq i\leq n$, such that the architect always chooses an architecture which maximizes the reported benefit of stakeholder $s_{i}$ and completely ignores the reported benefits provided by the remaining stakeholders $s_{j},j\neq i$.

We now illustrate Theorem 4.1 with two examples.

Example 1. Dictatorial CBAM. Consider a dictatorial version of CBAM, where the architect takes the benefit reported by stakeholder $s_{2}$ as the average reported benefit, and then selects the architecture with maximum desirability. Thus, the benefits reported by $s_{1}$ and $s_{3}$ are discarded by the architect and not used in the decision-making process.

In this selection method, there is no incentive for $s_{1}$ and $s_{3}$ to misreport their benefits, as they are anyway not part of the decision process. Thus, this method cannot be manipulated by $s_{1}$ and $s_{3}$. However, $s_{2}$ can still manipulate the method’s outcome, as shown by Table-3. If $s_{2}$ had reported her true benefit values, strategy $AS1$ with the desirability of $1.25$ would have been selected. (This is because, though $s_{2}$ has more benefit from $AS2$ than $AS1$, the cost of $AS1$ is much lower than that of $AS2$.) However, $s_{2}$ prefers $AS2$ more than $AS1$, and hence by misreporting a higher benefit of $100$ for $AS2$ and a lower benefit of $30$ for $AS1$, she ensures that $AS2$ has the highest desirability and gets selected instead of $AS1$.

Example 2. Plain Dictatorship. Now consider the same example as above, with the following change in selection algorithm $\mathcal{A}$. The benefits reported by $s_{1}$ and $s_{3}$ are discarded. However, now the architecture selected is the one with the maximum reported benefit for $s_{2}$.

Clearly, $s_{1}$ and $s_{3}$ have no incentive to misreport, as their feedback does not count. But now, $s_{2}$ also has no incentive to misreport her benefit values, as the strategy with maximum reported benefit value for $s_{2}$ always gets selected. Thus, this method is non-manipulable.

Thus, Theorem 4.1 puts serious limitations on CDM and shows that only software architecture selection methods similar to those in Example 2 above are non-manipulable. Any deviation from plain dictatorship (such as for example 1 above, or the example in §2.2), leads to scenarios where stakeholder(s) can unfairly influence the outcome in their favor.

5 Vickrey-Clarke-Groves (VCG) mechanism for software architecture selection

We now propose a novel non-manipulable software architecture selection method $\mathcal{M}$, using the well-known VCG mechanism [13],[3],[10]. The non-manipulability of the proposed approach is ensured by making payments to individual stakeholders; the payments are devised in such a way that a selfish stakeholder will always report her true benefit values.

Figure-3 depicts the proposed non-manipulable approach to selecting architecture using VCG. The numbers on the arrows in the figure depict the sequence of actions for selection. Our approach will always select the architecture with a maximum total reported benefit. Here, $u_{i}:X\rightarrow\mathbb{R},\forall i\in[1,n]$, denotes the actual benefits and $r_{i}:X\rightarrow\mathbb{R}$ denotes the reported benefits of stakeholder $s_{i}$.

The VCG module depicted in Figure-3 is discussed in Algorithm 1. For our algorithm (Algorithm 1) we use Clarke pivot rule 111Note that the payment calculation formula consists of two terms: the first term $T^{+}_{i}(r_{1},r_{2},\ldots,r_{n})$ depends on the benefits reported by stakeholder $s_{i}$; however, the second term $T^{-}_{i}(r_{1},r_{2},\ldots,r_{n})$ is due to Clarke pivot rule. This second term is independent of the benefits $r_{i}$ reported by stakeholder $s_{i}$. All payments in Clarke pivot rule are negative or zero. Hence, the architect can correspondingly decide the incentives. The reader is referred to [13], [3] for further details on Clarke pivot rule. with VCG.

5

5

5

5

5

As shown in Figure-3, after the VCG module is executed, the architect informs the stakeholders about the selected candidate architecture $\mathcal{VCG}(r_{1},r_{2},\ldots,r_{n})$ and further distributes net incentives equivalent to $p_{i}^{\mathcal{VCG}}(r_{1},r_{2},\ldots,r_{n})$ to each stakeholder $s_{i}$. Each stakeholder derives benefits due to two reasons: (i) an actual benefit of $u_{i}(\mathcal{VCG}(r_{1},r_{2},\ldots,r_{n}))$ due to the selection of the particular architecture by VCG module, and (ii) incentives to the value of $p_{i}^{\mathcal{VCG}}(r_{1},r_{2},\ldots,r_{n})$ from the architect. Thus, the net benefit $NB_{i}^{\mathcal{VCG}}(r_{1},r_{2},\ldots,r_{n})$ of stakeholder $s_{i}$ under $\mathcal{VCG}$ mechanism is equal to $u_{i}(a_{k^{*}})+p_{i}^{\mathcal{VCG}}(r_{1},r_{2},\ldots,r_{n})$. As the stakeholder is rational, her objective is now to maximize her net benefit. Thus, the following social choice theory theorem (Theorem-5.1), which is stated from [13][3], applies in this context.

Theorem 5.1

If the objective of each selfish stakeholder is to maximize her net benefit, then $\mathcal{VCG}$ is non-manipulable.

Here, the “net benefit” represents the stakeholder benefit after incorporating her incentive payments. Throughout this paper, we have used the term “benefits” to indicate just the direct benefits expected by a stakeholder when a specific architecture alternative gets selected.

It is pertinent here to note that if we replace “net benefit” with “benefit” in Theorem 5.1, then the mechanism becomes manipulable.

5.1 Examples of software architecture selection using $\mathcal{VCG}$ algorithm

We now illustrate the outcome of architecture selection by $\mathcal{VCG}$ on the examples considered earlier in this paper.

5.1.1 CBAM Example from §2.2.

We now implement $\mathcal{VCG}$ mechanism on the CBAM example given in §2.2.

Let $u_{1}$, $u_{2}$, and $u_{3}$ be the actual benefits of stakeholders $s_{1}$, $s_{2}$, and $s_{3}$ (see first three rows of Table-1). Further, let $r_{1}$, $r_{2}$, and $r_{3}$ denote the reported benefits of stakeholders $s_{1}$, $s_{2}$, and $s_{3}$ (see first three rows of Table-2). Table-5 describes the computation of payments $p_{i}^{\mathcal{VCG}}(r_{1},r_{2},r_{3})$ and net benefits $NB_{i}^{\mathcal{VCG}}(r_{1},r_{2},r_{3})$.

Now, suppose stakeholder $s_{1}$ unilaterally decides to report her true benefit values. Thus, the reported benefits are $u_{1}$, $r_{2}$, and $r_{3}$. The outcome of $\mathcal{VCG}$ mechanism is summarized in Table-5.

Thus, if stakeholder $s_{1}$ reports her benefit values truthfully to mechanism $\mathcal{VCG}$, and the other stakeholders do not change their benefit values, the net benefit of $s_{1}$ increases from $28$ to $65$. Hence, if the architect had implemented an algorithm $\mathcal{VCG}$, it would have been in the best interest of $s_{1}$ not to misreport her benefit values. Since $\mathcal{VCG}$ algorithm will ensure that each stakeholder speaks the truth, the architecture strategy $AS3$ will be selected by it since it has the maximum total reported (which would now be actual) benefit.

Note that in this particular example, the architecture strategy selected by $\mathcal{VCG}$ is the same as that selected by CBAM. However, this is not always the case.

5.1.2 Dictatorial CBAM example from §4.2.

Now consider the dictatorial CBAM method of §4.2. Further, suppose that the actual benefits $u_{2}$ and reported benefit $r_{2}$ of stakeholder $s_{2}$ are those given in first two rows of Table-3. To complete the input specification, we assume that the actual benefit vectors of $s_{1}$ and $s_{3}$ are $u_{1}=(55,~{}60,~{}10)$ and $u_{3}=(~{}40,~{}50,~{}-20)$ respectively. Table-7 describes the actual run of $\mathcal{VCG}$ for reported benefits $u_{1}$, $r_{2}$, and $u_{3}$.

Now suppose stakeholder $s_{2}$ had instead reported her true benefit values $u_{2}=(50,~{}80,-10)$. Table-7 describes a run of algorithm $\mathcal{VCG}$ with reported benefits $u_{1}$, $u_{2}$, and $u_{3}$. Since speaking the truth keeps the net benefit of $s_{2}$ the same, there is no incentive for her to lie under mechanism $\mathcal{VCG}$. Further, since all stakeholders are constrained to report true benefits, $\mathcal{VCG}(u_{1},u_{2},u_{3})$ is $AS2$, and this outcome is non-manipulable.

5.2 Discussion

5.2.1 Why a utilitarian objective function?

In algorithm $\mathcal{VCG}$ above, the architect always picks the architecture with maximum total reported benefit. The sum of all stakeholder benefits for a given architecture is called the utilitarian objective function. A natural question is whether VCG-type mechanisms can be devised which select an architecture based on objective functions other than the utilitarian objective function while ensuring truthfulness by rational stakeholders.

There is considerable literature in social choice theory and mechanism design on how to deal with this question. A theorem due to Roberts ([13]), proves that if there is no restriction on the possible benefit functions, then the only possible social choice functions for which the above VCG-type decision algorithm can force stakeholder truthfulness are weighted utilitarian functions. Essentially, these are social choice functions which always choose an architecture which maximizes the weighted sum of reported benefits of a particular subset of stakeholders. In light of these observations, the objective function chosen is utilitarian.

5.2.2 How can the architect incentivize the decision process?

Usually, the form of incentives are specific to an organization and its culture. For example, when all stakeholders are members of a single organization, the payments to be derived from the various stakeholders by algorithm $\mathcal{VCG}$ can be implemented by the administrators using absence or presence of certain organizational incentives such as higher salary, bonus, or shares, elevation or promotion, commendations, extra fund allocation, etc.

In a situation where some or all aspects of software development are being carried out by outside organizational entities (third party), the use of algorithm $\mathcal{VCG}$ by the architect will encourage truthfulness on part of various competing entities and lead to maximization of total benefits. The payments relative to outside entities stipulated by algorithm $\mathcal{VCG}$ can then be implemented using certain profitable contract clauses, long-term service contracts, recommendation for future projects, etc.

6 Related Work

To the best of our knowledge, there exists no decision-making technique in software architecture which uses mechanism design or social choice theory. Hence, we consider those methodologies related to our work which takes stakeholders’ concerns as input along with requirements and other environmental constraints.

In methods like CBAM [4], [1] and [5] the authors made the customer decide which stakeholder to be included in the decision-making process, and then these stakeholders give some numeric number or score to the quality attributes, where these number should sum to 100. They made the stakeholder describe why or how a particular quality is essential. To choose between the various architectural strategy they calculate a desirability factor which decreases with cost. CBAM paper lists three important criteria to arrive at an architectural decision: costs, benefits, and uncertainty or risks. All data gathered and used during an architecture decision process is at best only an estimate, and therefore is best represented by associated probability distributions, rather than exact, definite values. The CBAM framework measures the relevance of a candidate architecture by the ratio of benefits to costs.

In [7] the architects in collaborations with project stakeholders define the cost and benefit functions. Each alternative has a utility score defined as the weighted sum of the stakeholders’ preferences for each goal. Hence they measure the relevance of a candidate architecture by the “net benefit,” i.e., quantity obtained by subtracting the costs from the benefits. CEDA [12] considers stakeholder’s concerns to develop shared understanding aligned towards an organization’s goals. Tools like RADAR [2] use CBAM or ATAM as the starting point of developing an architecture and then handle particular aspects of architecture design decision like uncertainty using a Pareto optimal approach. Techniques like REARM[9] accept quantifiable input from stakeholders, but do not provide the details of how is to be done. Further, techniques like [8] integrate updated versions of CBAM-2 to handle structured analysis of architecture in agile development. [15] suggests a workshop based approach to handle concerns of stakeholders and specifically mentions as automating the handling of concerns and conflict as the need of the hour.

7 Conclusion and Future Work

Software architecture comprises of a set of decisions that need to be taken timely and correctly to avoid project failures. Reaching these decisions involves handling of the requirement of various stakeholders, who may (rightfully) push for their concerns to be realized in a certain way in software architecture. In software architecture research mostly this human aspect of architectural decisions is ignored.

We provide a new paradigm in software architecture decision-making by objectively handling human biases such as personal benefits, habits, strategically biased inputs, and so on. We identify the presence of rational stakeholders in the architectural decision-making process and propose the usage of social choice theory and mechanism design to handle their concerns. We provide a rigorous framework to ensure truthful information elicitation from rational stakeholders, thus ensuring that human factors do not influence software architecture design.

We are working on developing a tool, based on our proposed and validated VCG methodology to assist the architect in eliciting truthful information from stakeholders and thus make unbiased decisions while designing software architecture. This tool would be used by an architect to resolve the conflicts and concerns of stakeholders and automate the process of fair selection of architectural strategies.