As simple as possible but not simpler in Multiple Criteria Decision Aiding: the ROR-SMAA level dependent Choquet integral approach

TL;DR

This paper introduces a novel level dependent Choquet integral approach for multiple criteria decision aiding, incorporating robust and probabilistic analyses to handle multiple compatible models and decision maker preferences.

Contribution

It proposes two formulations of level dependent capacities and integrates ROR and SMAA to analyze all compatible models in decision making.

Findings

The methodology effectively ranks universities based on multiple criteria.

It accounts for uncertainty by considering all compatible capacities.

Probabilistic rankings provide nuanced decision support.

Abstract

The level dependent Choquet integral has been proposed to handle decision making problems in which the importance and the interaction of criteria may depend on the level of the alternatives' evaluations. This integral is based on a level dependent capacity, which is a family of single capacities associated to each level of evaluation for the considered criteria. We present two possible formulations of the level dependent capacity where importance and interaction of criteria are constant inside each one of the subintervals in which the interval of evaluations for considered criteria is split or vary with continuity inside the whole interval of evaluations. Since, in general, there is not only one but many level dependent capacities compatible with the preference information provided by the Decision Maker, we propose to take into account all of them by using the Robust Ordinal Regression (ROR) and the Stochastic Multicriteria Acceptability Analysis (SMAA). On one hand, ROR defines a necessary preference relation (if an alternative a is at least as good as an alternative b for all compatible level dependent capacities), and a possible preference relation (if a is at least as good as b for at least one compatible level dependent capacity). On the other hand, considering a random sampling of compatible level dependent capacities, SMAA gives the probability that each alternative reaches a certain ranking position as well as the probability that an alternative is preferred to another. A real-world decision problem on rankings of universities is provided to illustrate the proposed methodology.

Tables

Table 1. Table 1: Students’ notes on Mathematics and Physics

	Mathematics (M)	Physics (Ph)
$A$	28	28
$B$	30	26
$C$	26	30
$D$	23	23
$E$	25	21
$F$	21	25

Table 2. Table 2: Notes on Mathematics and Physics of the new students

	Mathematics (M)	Physics (Ph)
$G$	26	29
$H$	29	26
$I$	30	27

Table 3. Table 3: Description of the criteria

Criterion	Description
Bachelor Graduation Rate (BGR)	The percentage of new entrants that successfully completed their bachelor programme.
Masters Graduation Rate (MGR)	The percentage of new entrants that successfully completed their master programme.
Bachelors Graduation on Time (BGT)	The percentage of graduates that graduated within the time expected (normative time)
	for their bachelor programme.
Masters Graduation on Time (MGT)	The percentage of graduates that graduated within the time expected (normative time)
	for their masters programme.

Table 4. Table 4: Performances of universities on the Teaching and Learning aspect

	$BGR$	$MGR$	$BGT$	$MGT$
$U_1$	3	2	5	4
$U_2$	3	3	4	4
$U_3$	2	4	4	4
$U_4$	4	4	2	4
$U_5$	4	4	4	2
$U_6$	4	4	3	3
$U_7$	5	2	4	3
$U_8$	3	3	5	3
$U_9$	3	4	4	3
$U_{10}$	5	3	3	2
$U_{11}$	2	2	5	5
$U_{12}$	2	3	5	4
$U_{13}$	2	3	4	5
$U_{14}$	4	2	4	4
$U_{15}$	4	3	4	3
$U_{16}$	3	2	4	5
$U_{17}$	3	3	2	5
$U_{18}$	3	4	3	4
$U_{19}$	5	4	2	3

Table 5. (a) Sample of the universities

	$BGR$	$MGR$	$BGT$	$MGT$
$U_1$	3	2	5	4
$U_2$	3	3	4	4
$U_3$	2	4	4	4
$U_4$	4	4	2	4
$U_5$	4	4	4	2
$U_6$	4	4	3	3
$U_7$	5	2	4	3
$U_8$	3	3	5	3
$U_9$	3	4	4	3
$U_{10}$	5	3	3	2
$U_{11}$	2	2	5	5
$U_{12}$	2	3	5	4
$U_{13}$	2	3	4	5
$U_{14}$	4	2	4	4
$U_{15}$	4	3	4	3
$U_{16}$	3	2	4	5
$U_{17}$	3	3	2	5
$U_{18}$	3	4	3	4
$U_{19}$	5	4	2	3

Table 6. (b) Universities on which the DM provided his preferences

	$BGR$	$MGR$	$BGT$	$MGT$
$U_{20}$	5	5	5	4
$U_{21}$	3	2	2	3
$U_{22}$	5	4	4	5
$U_{23}$	3	3	3	2
$U_{24}$	5	4	5	4
$U_{25}$	4	5	5	4
$U_{26}$	2	3	3	3
$U_{27}$	2	3	3	2
$U_{28}$	3	2	3	2
$U_{29}$	4	5	5	5

Table 7. Table 5: Rank Acceptability indices for the considered universities

	$b^1(\cdot )$	$b^2(\cdot )$	$b^3(\cdot )$	$b^4(\cdot )$	$b^5(\cdot )$	$b^6(\cdot )$	$b^7(\cdot )$	$b^8(\cdot )$	$b^9(\cdot )$	$b^{10}(\cdot )$	$b^{11}(\cdot )$	$b^{12}(\cdot )$	$b^{13}(\cdot )$	$b^{14}(\cdot )$	$b^{15}(\cdot )$	$b^{16}(\cdot )$	$b^{17}(\cdot )$	$b^{18}(\cdot )$	$b^{19}(\cdot )$
$U_1$	0.261	5.511	10.258	11.567	10.474	7.405	6.247	5.365	6.219	5.29	5.227	5.156	4.837	4.231	4.486	2.852	3.194	1.157	0.263
$U_2$	0	0.221	1.448	5.285	9.186	12.281	13.971	13.33	12.842	9.735	9.824	6.901	3.388	0.987	0.561	0.04	0	0	0
$U_3$	0.001	0.013	0.098	0.228	0.555	0.823	0.795	1.151	1.884	2.156	2.545	3.939	5.492	7.25	11.736	16.019	16.734	15.598	12.983
$U_4$	0.45	1.852	2.335	1.683	2.719	2.583	3.106	3.748	3.968	4.863	4.541	5.329	6.314	6.468	8.35	9.766	12.042	11.055	8.828
$U_5$	11.409	8.189	8.489	8.105	8.125	6.906	6.405	6.381	5.205	4.287	4.4	4.198	4.872	5.222	3.994	2.44	0.964	0.409	0
$U_6$	0.156	0.971	3.005	5.686	5.5	6.283	6.032	5.909	5.958	7.523	7.421	7.51	8.734	9.421	7.345	6.571	5.547	0.397	0.031
$U_7$	4.925	3.732	5.354	6.485	7.326	6.617	5.931	5.843	4.752	4.337	4.504	5.032	5.056	6.198	7.919	7.65	4.699	2.852	0.788
$U_8$	2.165	3.094	4.395	5.777	6.566	6.134	5.965	5.845	5.678	5.37	5.83	5.823	6.939	7.416	6.837	6.163	4.64	3.421	1.942
$U_9$	1.125	3.865	4.489	5.659	5.754	6.034	6.983	7.679	8.133	9.077	10.572	11.091	9.294	6.573	2.707	0.859	0.106	0	0
$U_{10}$	0.002	0.172	0.216	0.175	0.366	0.273	0.399	0.709	1.29	1.914	2.046	2.16	2.707	4.187	6.327	10.301	14.982	17.199	34.575
$U_{11}$	10.098	15.65	15.175	8.586	5.391	3.628	3.652	2.8	3.178	3.021	3.333	3.424	3	3.662	3.735	2.622	1.932	3.936	3.177
$U_{12}$	0	0	0.06	0.385	0.746	2.378	2.871	3.163	3.294	3.67	3.97	5.435	6.117	6.853	7.994	10.135	12.956	14.737	15.236
$U_{13}$	2.881	14.07	13.639	9.138	6.819	5.829	4.784	4.147	3.979	4.195	3.783	3.955	4.597	4.541	3.964	4.899	3.477	1.075	0.228
$U_{14}$	1.478	14.957	10.974	9.822	8.362	7.491	6.417	5.906	4.77	5.028	4.337	4.8	4.625	4.565	3.436	1.962	0.543	0.225	0.302
$U_{15}$	0.006	0.177	1.07	3.279	6.106	8.651	10.31	10.36	12.741	13.109	10.313	9.528	6.599	4.574	2.677	0.451	0.049	0	0
$U_{16}$	60.842	17.604	7.103	3.182	2.226	1.239	0.964	1.042	1.095	1.104	1.031	0.881	0.984	0.578	0.125	0	0	0	0
$U_{17}$	1.109	4.534	4.778	7.42	4.529	3.817	3.886	3.516	3.06	3.545	4.27	4.143	6.505	7.759	10.223	8.077	7.266	6.046	5.517
$U_{18}$	1.231	3.438	6.183	6.546	8.101	9.509	9.011	10.324	9.453	9.473	9.081	7.237	5.337	3.862	1.117	0.084	0.013	0	0
$U_{19}$	1.861	1.95	0.932	0.991	1.15	2.118	2.271	2.785	2.5	2.302	2.971	3.458	4.603	5.653	6.467	9.109	10.856	21.894	16.129

Table 8. Table 6: Pairwise winning indices for the considered universities

	$U_1$	$U_2$	$U_3$	$U_4$	$U_5$	$U_6$	$U_7$	$U_8$	$U_9$	$U_{10}$	$U_{11}$	$U_{12}$	$U_{13}$	$U_{14}$	$U_{15}$	$U_{16}$	$U_{17}$	$U_{18}$	$U_{19}$
$U_1$	0	52.88	87.76	73.891	47.394	63.221	61.815	67.549	58.72	89.058	40.502	94.58	45.355	41.761	60.788	2.67	65.033	52.922	78.926
$U_2$	47.119	0	96.115	77.306	41.156	69.175	56.098	62.675	61.247	94.356	35.04	92.351	40.654	38.117	60.672	3.088	68.563	51.892	83.673
$U_3$	12.24	3.885	0	35.834	7.502	19.641	17.501	18.88	7.237	65.397	11.531	40.937	6.422	8.678	8.695	1.807	26.983	5.179	46.993
$U_4$	26.109	22.694	64.166	0	17.139	25.031	29.685	33.314	21.757	69.916	24.762	56.472	23.987	19.271	24.485	6.669	30.481	16.775	64.76
$U_5$	52.606	58.844	92.498	82.861	0	75.625	63.038	64.312	69.535	99.71	43.33	84.363	47.921	48.795	68.406	17.957	69.914	59.988	89.043
$U_6$	36.779	30.825	80.359	74.969	24.375	0	42.531	47.458	33.406	97.501	32.581	68.821	34.679	25.95	37.342	8.135	57.689	26.147	88.518
$U_7$	38.185	43.902	82.499	70.315	36.962	57.469	0	54.937	47.14	95.878	33.907	74.925	39.099	26.674	48.793	11.583	57.873	42.855	82.077
$U_8$	32.45	37.324	81.12	66.686	35.688	52.542	45.063	0	43.95	83.903	34.362	86.365	38.132	31.024	43.796	7.345	58.468	37.233	70.393
$U_9$	41.279	38.752	92.763	78.243	30.465	66.594	52.86	56.049	0	98.008	35.124	80.173	37.753	35.376	51.255	10.723	63.42	41.508	87.061
$U_{10}$	10.942	5.644	34.603	30.084	0.29	2.499	4.122	16.097	1.992	0	12.713	32.6	12.026	4.647	1.612	3.194	23.716	1.945	33.74
$U_{11}$	59.498	64.96	88.469	75.238	56.67	67.419	66.093	65.638	64.876	87.287	0	95.279	54.991	55.243	65.079	12.737	71.528	62.203	79.975
$U_{12}$	5.42	7.649	59.062	43.528	15.637	31.179	25.075	13.635	19.827	67.4	4.721	0	2.67	13.532	17.485	0.309	31.26	16.342	52.378
$U_{13}$	54.645	59.346	93.578	76.013	52.079	65.321	60.901	61.868	62.247	87.974	45.009	97.33	0	50.715	61.209	5.42	73.457	58.617	79.244
$U_{14}$	58.239	61.883	91.322	80.729	51.204	74.05	73.326	68.976	64.624	95.353	44.757	86.468	49.285	0	68.872	10.233	67.769	59.295	86.586
$U_{15}$	39.212	39.328	91.305	75.515	31.594	62.658	51.206	56.204	48.745	98.388	34.921	82.515	38.791	31.128	0	7.6	63.462	38.995	82.695
$U_{16}$	97.33	96.912	98.193	93.331	82.043	91.865	88.417	92.655	89.277	96.806	87.263	99.691	94.58	89.767	92.4	0	97.547	89.933	92.688
$U_{17}$	34.967	31.437	73.017	69.519	30.086	42.311	42.127	41.532	36.58	76.284	28.472	68.74	26.543	32.231	36.538	2.453	0	30.261	69.217
$U_{18}$	47.078	48.108	94.821	83.225	40.012	73.853	57.145	62.767	58.492	98.055	37.797	83.658	41.383	40.705	61.005	10.067	69.739	0	87.711
$U_{19}$	21.074	16.327	53.007	35.24	10.957	11.482	17.923	29.607	12.939	66.26	20.025	47.622	20.756	13.414	17.305	7.312	30.783	12.289	0

Table 9. Table 7: Best, worst and most frequent positions

	Best	$b^{West}(\cdot )$	Worst	$b^{Worst}(\cdot )$
$U_1$	1	0.261	19	0.263
$U_2$	2	0.221	16	0.04
$U_3$	1	0.001	19	12.983
$U_4$	1	0.45	19	8.828
$U_5$	1	11.409	18	0.409
$U_6$	1	0.156	19	0.031
$U_7$	1	4.925	19	0.788
$U_8$	1	2.165	19	1.942
$U_9$	1	1.125	17	0.106
$U_{10}$	1	0.002	19	34.575
$U_{11}$	1	10.098	19	3.177
$U_{12}$	3	0.06	19	15.236
$U_{13}$	1	2.881	19	0.228
$U_{14}$	1	1.478	19	0.302
$U_{15}$	1	0.006	17	0.049
$U_{16}$	1	60.842	15	0.125
$U_{17}$	1	1.109	19	5.517
$U_{18}$	1	1.231	17	0.013
$U_{19}$	1	1.861	19	16.129

Table 10. (a) Best and worst positions

	Best	$b^{West}(\cdot )$	Worst	$b^{Worst}(\cdot )$
$U_1$	1	0.261	19	0.263
$U_2$	2	0.221	16	0.04
$U_3$	1	0.001	19	12.983
$U_4$	1	0.45	19	8.828
$U_5$	1	11.409	18	0.409
$U_6$	1	0.156	19	0.031
$U_7$	1	4.925	19	0.788
$U_8$	1	2.165	19	1.942
$U_9$	1	1.125	17	0.106
$U_{10}$	1	0.002	19	34.575
$U_{11}$	1	10.098	19	3.177
$U_{12}$	3	0.06	19	15.236
$U_{13}$	1	2.881	19	0.228
$U_{14}$	1	1.478	19	0.302
$U_{15}$	1	0.006	17	0.049
$U_{16}$	1	60.842	15	0.125
$U_{17}$	1	1.109	19	5.517
$U_{18}$	1	1.231	17	0.013
$U_{19}$	1	1.861	19	16.129

Table 11. (b) Most frequent positions

	$hig{h}_1$	$b^{hig{h}_1}(\cdot )$	$hig{h}_2$	$b^{hig{h}_2}(\cdot )$	$hig{h}_3$	$b^{hig{h}_3}(\cdot )$
$U_1$	4	11.567	5	10.474	3	10.258
$U_2$	7	13.971	8	13.33	9	12.842
$U_3$	17	16.734	16	16.019	18	15.598
$U_4$	17	12.042	18	11.055	16	9.766
$U_5$	1	11.409	3	8.489	2	8.189
$U_6$	14	9.421	13	8.734	10	7.523
$U_7$	15	7.919	16	7.65	5	7.326
$U_8$	14	7.416	13	6.939	15	6.837
$U_9$	12	11.091	11	10.572	13	9.294
$U_{10}$	19	34.575	18	17.199	17	14.982
$U_{11}$	2	15.65	3	15.175	1	10.098
$U_{12}$	19	15.236	18	14.737	17	12.956
$U_{13}$	2	14.07	3	13.639	4	9.138
$U_{14}$	2	14.957	3	10.974	4	9.822
$U_{15}$	10	13.109	9	12.741	8	10.36
$U_{16}$	1	60.842	2	17.604	3	7.103
$U_{17}$	15	10.223	16	8.077	14	7.759
$U_{18}$	8	10.324	6	9.509	10	9.473
$U_{19}$	18	21.894	19	16.129	17	10.856

Table 12. Table 8: Expected ranking of the universities at hand

University	$E(\cdot )$	Rank-position
$U_{16}$	-229.302	1
$U_{11}$	-706.817	2
$U_{14}$	-707.028	3
$U_5$	-711.253	4
$U_{13}$	-755.027	5
$U_{18}$	-804.379	6
$U_1$	-815.172	7
$U_2$	-820.7	8
$U_9$	-902.591	9
$U_{15}$	-925.737	10
$U_7$	-954.926	11
$U_8$	-1014.15	12
$U_6$	-1051.94	13
$U_{17}$	-1127.69	14
$U_4$	-1322.53	15
$U_{19}$	-1455.68	16
$U_{12}$	-1472.89	17
$U_3$	-1554.66	18
$U_{10}$	-1667.53	19

Equations

$$A\succ C\succ B\succ E\succ F\succ D$$

$$U(X)=w_M\cdot M(X)+w_{Ph}\cdot Ph(X)$$

$$U(B)=w_M\cdot 30+w_{Ph}\cdot 26<w_M\cdot 26+w_{Ph}\cdot 30=U(C)$$

$$U(E)=w_M\cdot 25+w_{Ph}\cdot 21>w_M\cdot 21+w_{Ph}\cdot 25=U(F)$$

$$w_M^1,w_M^2,w_{Ph}^1,w_{Ph}^2⩾0,w_M^1+w_{Ph}^1=1,w_M^2+w_{Ph}^2=1.$$

$$U_{LDWS}(X)=w_M^1\cdot M^1(X)+w_M^2\cdot M^2(X)+w_{Ph}^1\cdot P{h}^1(X)+w_{Ph}^2\cdot P{h}^2(X)$$

$$U_{LDWS}(X)=w_M^1\cdot \min \{M(X),25\}+w_M^2\cdot \max \{M(X)-25,0\}+w_{Ph}^1\cdot \min \{Ph(X),25\}+$$

$$+w_{Ph}^2\cdot \max \{Ph(X)-25,0\}.$$

$$U_{LDWS}(B)=w_M^1\cdot \min \{30,25\}+w_M^2\cdot \max \{30-25,0\}+w_{Ph}^1\cdot \min \{26,25\}+w_{Ph}^2\cdot \max \{26-25,0\}$$

$$<$$

$$w_M^1\cdot \min \{26,25\}+w_M^2\cdot \max \{26-25,0\}+w_{Ph}^1\cdot \min \{30,25\}+w_{Ph}^2\cdot \max \{30-25,0\}=U_{LDWS}(C)$$

$$U_{LDWS}(E)=w_M^1\cdot \min \{25,25\}+w_M^2\cdot \max \{25-25,0\}+w_{Ph}^1\cdot \min \{21,25\}+w_{Ph}^2\cdot \max \{21-25,0\}$$

$$>$$

$$w_M^1\cdot \min \{21,25\}+w_M^2\cdot \max \{21-25,0\}+w_{Ph}^1\cdot \min \{25,25\}+w_{Ph}^2\cdot \max \{25-25,0\}=U_{LDWS}(F).$$

$$w_M^1\cdot 25+w_M^2\cdot 5+w_{Ph}^1\cdot 25+w_{Ph}^2\cdot 1<w_M^1\cdot 25+w_M^2\cdot 1+w_{Ph}^1\cdot 25+w_{Ph}^2\cdot 5$$

$$w_M^1\cdot 25+w_M^2\cdot 0+w_{Ph}^1\cdot 21+w_{Ph}^2\cdot 0>w_M^1\cdot 21+w_M^2\cdot 0+w_{Ph}^1\cdot 25+w_{Ph}^2\cdot 0,$$

$$U_{LDWS}(A)>U_{LDWS}(C)>U_{LDWS}(B).$$

$$w_M^1\cdot \min \{28,25\}+w_M^2\cdot \max \{28-25,0\}+w_{Ph}^1\cdot \min \{28,25\}+w_{Ph}^2\cdot \max \{28-25,0\}$$

$$>$$

$$w_M^1\cdot \min \{26,25\}+w_M^2\cdot \max \{26-25,0\}+w_{Ph}^1\cdot \min \{30,25\}+w_{Ph}^2\cdot \max \{30-25,0\}$$

$$>$$

$$w_M^1\cdot \min \{30,25\}+w_M^2\cdot \max \{30-25,0\}+w_{Ph}^1\cdot \min \{26,25\}+w_{Ph}^2\cdot \max \{26-25,0\}$$

$$U_{LDWS}(E)>U_{LDWS}(F)>U_{LDWS}(D).$$

$$U_{Ch}(X)=\left\{\begin{array}[]{lll}M(X)+w^{Ch}_{Ph}\cdot(Ph(X)-M(X))&\mbox{iff}&M(X)\leqslant Ph(x),\\[1.42262pt] Ph(X)+w^{Ch}_{M}\cdot(M(X)-Ph(X))&\mbox{iff}&M(X)\geqslant Ph(x),\\[1.42262pt] \end{array}\right.$$

$$U_{Ch}(A)>U_{Ch}(C)>U_{Ch}(B),$$

$$28+w_{Ph}^{Ch}\cdot 0>26+4\cdot w_{Ph}^{Ch}>26+4\cdot w_M^{Ch}$$

$$0.5>w_{Ph}^{Ch}>w_M^{Ch}$$

$$U_{Ch}(E)>U_{Ch}(F)>U_{Ch}(D)$$

$$21+4\cdot w_M^{Ch}>21+4\cdot w_{Ph}^{Ch}>23+0\cdot w_{Ph}^{Ch}.$$

Full text

As simple as possible but not simpler in Multiple Criteria Decision Aiding: the ROR-SMAA level dependent Choquet integral approach

Sally Giuseppe Arcidiacono

s.arcidiacono@unict.it

Salvatore Corrente

salvatore.corrente@unict.it

Salvatore Greco

salgreco@unict.it

Department of Economics and Business, University of Catania, Corso Italia, 55, 95129 Catania, Italy

Portsmouth Business School, Centre for Operational Research and Logistics (CORL), University of Portsmouth, Portsmouth, United Kingdom

• Abstract: The level dependent Choquet integral has been proposed to handle decision making problems in which the importance and the interaction of criteria may depend on the level of the alternatives’ evaluations. This integral is based on a level dependent capacity, which is a family of single capacities associated to each level of evaluation for the considered criteria. We present two possible formulations of the level dependent capacity where importance and interaction of criteria are constant inside each one of the subintervals in which the interval of evaluations for considered criteria is split or vary with continuity inside the whole interval of evaluations. Since, in general, there is not only one but many level dependent capacities compatible with the preference information provided by the Decision Maker, we propose to take into account all of them by using the Robust Ordinal Regression (ROR) and the Stochastic Multicriteria Acceptability Analysis (SMAA). On one hand, ROR defines a necessary preference relation (if an alternative $a$ is at least as good as an alternative $b$ for all compatible level dependent capacities), and a possible preference relation (if $a$ is at least as good as $b$ for at least one compatible level dependent capacity). On the other hand, considering a random sampling of compatible level dependent capacities, SMAA gives the probability that each alternative reaches a certain ranking position as well as the probability that an alternative is preferred to another. A real-world decision problem on rankings of universities is provided to illustrate the proposed methodology.

Keywords: Decision support systems, Level dependent Choquet integral, Robust Ordinal Regression, Stochastic Multicriteria Acceptability Analysis, Ranking of University

1 Introduction

Decision support is generally based on formal mathematical models representing the preferences of the Decision Maker (DM). Since all formal mathematical models are a simplified representation of the phenomena one wants to investigate, a decision support model is a simplified representation of a process that the DM can adopt to take a sensible decision. The idea of simplicity of a model is often related to the Ockham’s razor, usually expressed as “pluralitas non est ponenda sine necessitate” (plurality should not be posited without necessity) even if in his “Summa Totius Logicae”, William of Ockham wrote in a different way: “Frustra fit per plura quod potest fieri per pauciora” (it is futile to do with more things that which can be done with fewer) [27, 47]. The search of simplicity is, for sure, an important point of any scientific model. However it has to be tempered with the idea that the model should represent all the important elements that can explain the phenomenon under investigation. In this sense, there is another very well known principle that is generally attributed to Albert Einstein: “Everything should be made as simple as possible, but not simpler”. In fact, the exact expression of Einstein is the following: “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience” [13]. We believe that this principle has a specific importance for decision support models. They aim to provide a recommendation that has to be understandable and convincing for the DM and not an ideally abstract description of the decision processes as can be the case of the decision models considered in economics [39]. Indeed, in economics, following the considerations of Milton Friedman [14], a model has to be evaluated in terms of simplicity (that is minimality of initial knowledge needed) and fruitfulness (that is accuracy and generality of the obtained prediction). Instead, in decision support, a model has to be appreciated in terms of the interaction it permits with the DM and in terms of the convincing arguments it produces [40]. With this aim, the focus shifts from the simplicity to the richness of the model that, within decision support, is seen as a feature that facilitates the interaction with the DM that can see the model as a language to express his preferences with the desired degree of detail.

On the basis of these considerations, in this paper we propose a quite “rich” decision support model that can be seen as a relevant application and an exemplification of the tempered parsimony we have discussed. Our model is constituted by the Robust Ordinal Regression (ROR; see [22] for the seminal paper and [8, 9] for two comprehensive surveys) and Stochastic Multicriteria Acceptability Analysis (SMAA; [28] for the seminal paper and [30, 37] for two comprehensive surveys) applied to the level dependent Choquet integral [21]. The decision analysis model so obtained permits to take into consideration the three following relevant aspects of a multicriteria decision making problem:

interaction between criteria,

• 2.

importance and interaction of criteria changing from one level to another of the evaluations in the considered criteria,

• 3.

robustness concerns with respect to the value assigned to the parameters of the decision model.

Let us discuss more in detail each one of the above points. Interaction between criteria refers to the phenomenon for which when aggregating their evaluations to obtain an overall assessment, criteria can reinforce or can weaken each other producing a synergy or a redundancy, respectively; following a very well known example [15], evaluating students of a technical school we can have:

a redundancy effect between Mathematics and Physics: students good in Mathematics are, in general, good also in Physics. Therefore, to avoid an over-evaluation of students good in both subjects, Mathematics and Physics considered together should have a weight smaller than the sum of their weights when considered separately;

• 2.

a synergy effect between Mathematics and Literature: students good in Mathematics are, in general, not good in Literature. Therefore, a student presenting good marks in both subjects is well appreciated and a special bonus is considered in evaluating him. Consequently, Mathematics and Literature considered together should have a weight greater than the sum of their weights when considered separately.

We have to observe also that importance and interaction of criteria can change from one level to another of the evaluations in the considered criteria; again, in evaluating students in a technical school, we can have the following:

the importance of Literature can be greater than that one of Mathematics and Physics in evaluating students with rather low notes on the last two subjects: indeed, a relatively good note in Literature means that there is some potential for the student to improve despite the low notes in Mathematics and Physics,

• 2.

the importance of Literature is not so great in evaluating students with medium notes in all subjects: in this case, the focus is more on the core subjects of the school that are Mathematics and Physics,

• 3.

the importance of Literature is again very high in evaluating students with high notes in all subjects: in this case, one expects excellence and this is obtained if the students are good in all subjects and not only in Mathematics and Physics;

• 4.

Mathematics and Physics can present a synergetic effect in evaluating students with low notes: indeed, in this case, a relatively good note in both subjects means that the student can improve;

• 5.

Mathematics and Physics can present a redundant effect in evaluating students with medium notes: in this case, one relatively good evaluation in one subject is already considered enough;

• 6.

Mathematics and Physics can present a synergetic effect in evaluating students with high notes in both subjects: in this case, good evaluations are necessary to be evaluated excellent.

Another important issue in any decision aiding procedure is the robustness of the final recommendation. More in detail, any formal decision model requires fixing the value of a certain number of parameters. Of course, perturbation of the values of these parameters can bring to different results. Consequently, to get a sufficiently reliable recommendation, it is necessary to explore the stability of the suggestions proposed by the adopted decision model changing the values of its parameters.

The decision model we are considering is able to take into account the above three issues in the way we are going to present in the following.

On one hand, the level dependent Choquet integral permits to handle criteria for which the importance and the interaction change from one level to the other of the evaluations. Indeed, the Choquet integral permits to aggregate the evaluations taking into account interaction of criteria by using capacities that assign a weight to each subset of the considered criteria and not only to single criteria as in the usual weighted sum. The level dependent Choquet integral goes a further step forward taking into account a set of level dependent capacities, that is, a capacity for each level of the evaluation criteria. This permits to represent different attitudes, that is, different importance and interaction assigned to considered criteria, for different evaluation levels (for an alternative decision model permitting to represent interaction of criteria that changes from one level to another of the evaluations see [23]).

On the other hand, the robustness concerns are handled by an exploration of the space of the feasible parameters of the decision model, that are, the level dependent capacities compatible with the preference information supplied by the DM. This is done by applying:

ROR, that, for each pair of alternatives $a$ and $b$, tells us if $a$ is preferred to $b$ for all compatible level dependent capacities (in which case, we say that $a$ is necessarily preferred to $b$) or for at least one compatible level dependent capacity (in which case we say that $a$ is possibly preferred to $b$;

• 2.

SMAA, that, for each alternative $a$ and for each rank order position $r$, tells us which is the probability that, randomly extracting a compatible level dependent capacity, $a$ attains the position $r$, as well as, for each pair of alternatives $a$ and $b$, the probability that $a$ is preferred to $b$.

Another interesting issue to be considered is the modeling of changes of importance and interaction of criteria from one level to another of the evaluations on considered criteria. In fact, two main approaches could be considered:

partitioning in intervals separated by reference levels range of possible evaluations on considered criteria, adopting preference parameters representing importance and interaction between criteria that change from one interval to another, but that remain constant within the same interval;

• 2.

using preference parameters representing importance and interaction between criteria that change continuously from one level of evaluations to another without jumps.

The first approach is clearly more manageable and could be considered a reasonable assumption to model preferences. In this way, the model can be refined by increasing the number of intervals of the partition and, therefore, representing better the changes of importance and interaction of criteria from one level of evaluation to another. This type of preference representation can be formulated in terms of Choquet integral based on interval level dependent capacities proposed in [21].

The second approach could seem more intuitive: it is reasonable to imagine that the importance and interaction of criteria change smoothly without sudden jumps from one level of evaluation to another. Ideally, a model of this type has a set of preference parameters modeling importance and interaction of criteria for each level of evaluations. This is clearly a problem from the point of view of induction of these parameters from preference information supplied by the DM and their effective use in decision support. Therefore, if one wants that importance and interaction of criteria change continuously, he has to consider some model that represents these changes taking into account a not so big number of parameters that can be induced and handled with a quite manageable procedure, for example, by linear programming. With this aim, we propose the Choquet integral based on piecewise linear level dependent capacities. By using this model, the parameters representing importance and interaction between criteria at each level of evaluation can be determined as linear interpolation of the values that they assume in specific reference levels. We shall show that also the Choquet integral can be formulated as a linear function of the values assumed by the preference parameters in the reference levels, so that linear programming techniques and sampling procedures can still be used in applying ordinal regression, ROR and SMAA. Also this Choquet integral based on piecewise linear level dependent capacities that we are proposing to handle continuous changes in the preference parameters, must be seen in the context of tempered parsimony of multiple criteria decision aiding models that we are advocating. In synthetic terms, we can say that this model can be seen as the exemplification of a principle dual to the above mentioned Einstein principle of parsimony: “Everything should be made as realistic as possible, but not more complex”.

The paper is organized as follows. In the next section, we present an introductory example that, step by step, informally introduces the model with its potentialities. In the third section, we recall the Choquet integral as well as the level dependent Choquet integral based on interval level dependent capacities, while, in the fourth section, we present the application of ROR and SMAA to this decision model. The fifth section presents the Choquet integral based on piecewise linear capacities permitting to represent continuous changes in importance and interaction of criteria. In the sixth section, a real world application of the level dependent Choquet integral to the ranking of universities is described. The last section collects conclusions and perspectives for future developments.

2 An introductory example

Let us explain the richness and usefulness of the level dependent Choquet integral coupled with methodologies taking into account robustness concerns, namely ROR and SMAA, and representing quite articulated preferences of a DM. Ideally this example illustrates the basic ideas of the tempered parsimony we are postulating, so that it is presented with a certain degree of detail. Inspired by [15], let us suppose that the dean of a scientifically oriented high school wants to provide an overall evaluation of its students. For the sake of simplicity, the dean wants to take into account only notes on Mathematics and Physics, expressed on a thirty point scale. The dean expresses the following convictions:

considering the required skills, Physics is more important than Mathematics in evaluating students with good notes on both subjects, that are those having a mark of at least 26 in these subjects;

• 2.

considering students needing to improve their background, that are students having a mark at most equal to 25 on the two subjects, Mathematics is more important than Physics;

• 3.

a student with notes at least equal to 26 in both subjects is excellent if he has very good notes both in Mathematics and in Physics;

• 4.

among students with notes at most equal to 25 in both subjects, the dean prefers students with a relatively good note in at least one of the subjects, because this can be seen as a starting point to reduce the gap from better students.

On the basis of the above considerations, taking into account the students whose evaluations on the two subjects are presented in Table 1, the dean expresses the following preferences

[TABLE]

where $X\succ Y$ means that student $X$ is preferred to student $Y$, with $X,Y\in\{A,B,C,D,E,F\}$.

One can see that the dean is giving a rich preference information, but one can also wonder if it is possible to represent his preferences using a model as simple as possible. The simplest model one can consider is the weighted sum, that is

[TABLE]

where

$U(X)$ is the overall evaluation of $X$,

• 2.

$M(X)$ and $Ph(X)$ are the notes of $X$ in Mathematics and Physics, respectively,

• 3.

$w_{M}$ and $w_{Ph}$ are the weights of Mathematics and Physics and they are such that, $w_{M}\geqslant 0$, $w_{Ph}\geqslant 0$ and $w_{M}+w_{Ph}=1$.

Apparently, a so simple model cannot model the preference information supplied by the dean. On one hand, taking into account students $B$ and $C$ and reminding that the dean prefers $C$ over $B$, one gets,

[TABLE]

implying that $w_{M}<w_{Ph}$. On the other hand, considering $E$ and $F$ and reminding that the dean prefers $E$ over $F$, one gets

[TABLE]

that implies $w_{M}>w_{Ph}$. Of course the two inequalities translating the preferences of the dean are incompatible and, consequently, the weighted sum is not able to represent these preferences.

Looking for an amendment of the weighted sum able to represent preferences that cannot be expressed by using (1), one could imagine that weights depend on the level of the notes. For example, one can imagine that Mathematics has a weight $w^{1}_{M}$ for notes up to 25, that is $M^{1}(X)=min\{M(X),25\}$, and weight $w^{2}_{M}$ for the possible excess to 25, that is, $M^{2}(X)=\max\{M(X)-25,0\}$. Analogously, one could assume that Physics has weight $w^{1}_{Ph}$ for notes up to 25, that is, $Ph^{1}(X)=\min\{Ph(X),25\}$, and weight $w^{2}_{Ph}$ for the possible excess to 25, that is, $Ph^{2}(X)=\max\{Ph(X)-25,0\}$. Nonnegativity and normalization of the weights in this case would require that

[TABLE]

In this way, we could have a new model, the level dependent weighted sum, (in fact, the two additive level dependent weighted sum), having the following form:

[TABLE]

that is

[TABLE]

[TABLE]

The level dependent weighted sum permits to take into account that, for notes at least equal to 26, Physics is more important than Mathematics ($w^{2}_{Ph}>w^{2}_{M}$), while for notes at most equal to 25, it is exactly the opposite ($w^{1}_{M}>w^{1}_{Ph}$).

The level dependent weighted sum is therefore able to represent some of the preferences provided by the dean. Indeed, considering students $B,C,E$ and $F$, the preference of $C$ over $B$ is translated to the constraint

[TABLE]

[TABLE]

[TABLE]

while, the preference of $E$ over $F$ is translated to the constraint

[TABLE]

[TABLE]

[TABLE]

On one hand, by (2) one gets

[TABLE]

and, consequently, $w^{2}_{M}<w^{2}_{Ph}$.

On the other hand, by (3) one gets

[TABLE]

and, consequently, $w^{1}_{M}>w^{1}_{Ph}$.

Therefore, we have seen that the level dependent weighted sum permits to represent importance of criteria changing from one level to another. However, we have still to check if $U_{LDWS}$ allows to consider also the conjunctive or disjunctive nature of the overall evaluation that changes from one level to another. In fact, according with the preference information supplied by the dean, the aggregation procedure has to be:

conjunctive for notes at least equal to 26 in both subjects: in this case, a student good in both subjects is more appreciated than a student very good in one subject and not so good in the other,

• 2.

disjunctive for notes at most equal to 25 in both subjects: in this case, a student relatively good in at least one of the two subjects is more appreciated than a student presenting medium marks on both subjects.

In consequence of the previous observations, in evaluating students $A$, $B$ and $C$, the dean stated that $A$ is preferred to $C$ that, in turn, is preferred to $B$. Using the $U_{LDWS}$, the previous preferences are translated into the constraints

[TABLE]

The first inequality, that is, $U_{LDWS}(A)>U_{LDWS}(C)$, implies that

[TABLE]

[TABLE]

[TABLE]

from which, $w^{2}_{M}>w^{2}_{Ph}$.

The second inequality, that is $U_{LDWS}(C)>U_{LDWS}(B)$, instead, implies that

[TABLE]

[TABLE]

[TABLE]

and, consequently, $w^{2}_{Ph}>w^{2}_{M}$. Of course, the two obtained inequalities are incompatible since $w^{2}_{Ph}$ cannot be simultaneously lower and greater than $w^{2}_{M}$.

Analogously, taking into consideration students $D,E$ and $F$, and reminding that the dean stated that $E$ is preferred to $F$ that, in turn, is preferred to $D$, the application of the level dependent weighted sum would translate these preferences into the following chain of constraints

[TABLE]

On one hand, the first inequality, that is $U_{LDWS}(E)>U_{LDWS}(F)$, implies $w^{1}_{M}>w^{1}_{Ph}$, while, on the other hand, the second inequality, that is $U_{LDWS}(F)>U_{LDWS}(D)$, implies that $w^{1}_{M}<w^{1}_{Ph}$. Again, the two inequalities are incompatible and, consequently, even the level dependent weighted sum is not able to represent the preferences given by the DM.

To represent the more or less conjunctive character of the aggregation procedure one can use the Choquet integral [7] being a generalization of the weighted sum that has been successfully applied in multiple criteria decision analysis to handle interaction between criteria. The Choquet integral of the evaluations of student $X$ on Mathematics and Physics can be defined as follows:

[TABLE]

with $0\leqslant w^{Ch}_{M},w^{Ch}_{Ph}\leqslant 1$ and not necessarily $w^{Ch}_{M}+w^{Ch}_{Ph}=1$. Let us see if the Choquet integral permits to represent the preferences expressed by the dean on students $A,B$ and $C$. Since $A\succ C\succ B$, we should have

[TABLE]

and, consequently,

[TABLE]

from which

[TABLE]

which is in agreement with the greater importance assigned by the dean to Physics over Mathematics in case of notes not smaller than 26 in both subjects. Thus we proved that the Choquet integral can represent the preferences about students $A,B$ and $C$.

With analogous considerations, we can show that the Choquet integral can represent also the preferences over students $D,E$ and $F$. In fact, from $E\succ F\succ D$ we obtain

[TABLE]

and, consequently,

[TABLE]

The inequalities above imply

[TABLE]

which are in agreement with the greater importance assigned by the dean to Mathematics over Physics in case of notes not greater than 25 in both subjects. Thus also the preferences between the students $E,F$ and $D$ can be represented by the Choquet integral $U_{Ch}$.

However, let us remark that conditions (4) and (5) are incompatible, which means that the Choquet integral is not able to represent, simultaneously, all the preferences expressed by the dean over the six students. The problem is that the importance of criteria as well as the conjunctive or disjunctive nature of the aggregation procedure are different for students with notes at least equal to 26 and students with notes at most equal to 25. Thus, one has to further generalize the model considering the level dependent Choquet integral [21] that permits to take into account importance of criteria and interaction between criteria related to conjunctive or disjunctive nature of the aggregation procedure that can change from one level to the others of the evaluations on considered criteria. With respect to the overall evaluation of students on Mathematics and Physics, the level dependent Choquet integral can be formulated as follows:

[TABLE]

where

[TABLE]

and

[TABLE]

Let us check if the level dependent Choquet integral is able to represent the preferences over the six students, that is if there exist $w_{M}^{Ch,1}$, $w_{M}^{Ch,2}$, $w_{Ph}^{Ch,1}$ and $w_{Ph}^{Ch,2}$ such that

[TABLE]

Using (6), the previous inequalities are translated into the following chain of constraints:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

We obtain the following conditions:

$U_{LDCh}(A)>U_{LDCh}(C)$ $\Rightarrow$ $2>w^{Ch,2}_{Ph}\cdot 4$ $\Rightarrow$ $0.5>w^{Ch,2}_{Ph}$, 2. 2.

$U_{LDCh}(C)>U_{LDCh}(B)$ $\Rightarrow$ $w^{Ch,2}_{Ph}\cdot 4>w^{Ch,2}_{M}\cdot 4$ $\Rightarrow$ $w^{Ch,2}_{Ph}>w^{Ch,2}_{M}$, 3. 3.

$U_{LDCh}(E)>U_{LDCh}(F)$ $\Rightarrow$ $w^{Ch,1}_{M}\cdot 4>w^{Ch,1}_{Ph}\cdot 4$ $\Rightarrow$ $w^{Ch,1}_{M}>w^{Ch,1}_{Ph}$, 4. 4.

$U_{LDCh}(F)>U_{LDCh}(D)$ $\Rightarrow$ $w^{Ch,1}_{Ph}\cdot 4>2$ $\Rightarrow$ $w^{Ch,1}_{Ph}>0.5$.

The four inequalities above are not in contradiction and, therefore, the level dependent Choquet integral permits to represent the preferences expressed by the dean. Moreover, the conditions perfectly translate the preferences of the dean in terms of importance of criteria and the desired conjunctive and disjunctive nature of the aggregation. However, there is still another aspect that deserves to be investigated. Let us consider the new students $G$, $H$ and $I$ whose notes in Mathematics and Physics are presented in Table 2 and let us try to rank them on the basis of conditions 1.-4. above.

On one hand, comparing $G$ and $H$, we get that

[TABLE]

[TABLE]

[TABLE]

being always true since it coincides with condition 2.

On the other hand, comparing $G$ and $I$, we can have $U_{LDCh}(G)>U_{LDCh}(I)$ as well as $U_{LDCh}(I)>U_{LDCh}(G)$, depending on the values of $w_{M}^{Ch,1}$, $w_{M}^{Ch,2}$, $w_{Ph}^{Ch,1}$ and $w_{Ph}^{Ch,2}$. Indeed, we get

[TABLE]

[TABLE]

[TABLE]

iff

[TABLE]

as well as $U_{LDCh}(G)<U_{LDCh}(I)$ iff

[TABLE]

None of the two inequalities is in contradiction with conditions 1.-4. Therefore, we have to conclude that the preference information given by the DM, and formally expressed by conditions 1.-4., does not permit to rank order unambiguously all students since different values of parameters can provide a different comparison of $G$ and $I$. For example, taking $w^{Ch,2}_{M}=\frac{1}{18}$ and $w^{Ch,2}_{Ph}=\frac{4}{9}$ we get

[TABLE]

while, taking $w^{Ch,2}_{M}=\frac{1}{6}$ and $w^{Ch,2}_{Ph}=\frac{1}{3}$ we get

[TABLE]

With a greater level of detail, we can say that student $G$ will be overall evaluated at least as good as student $I$, that is, $U_{LDCh}(G)\geqslant U_{LDCh}(I)$, in case $w^{Ch,2}_{M}$ and $w^{Ch,2}_{Ph}$ satisfy conditions 1.-4. with the addition of the following one

[TABLE]

In the space $w^{Ch,2}_{M}-w^{Ch,2}_{Ph}$, the preference parameters compatible with this piece of preference information are represented by the points in the triangle $W_{G\succsim I}$ having vertices $P_{2}\equiv(\frac{1}{6},\frac{1}{2})$, $P_{3}\equiv(0,\frac{1}{2})$ and $P_{4}\equiv(0,\frac{1}{3})$ in Figure 1. Instead, student $I$ will get an overall evaluation not worse than $G$, that is, $U_{LDCh}(I)\geqslant U_{LDCh}(G)$, in case $w^{Ch,2}_{M}$ and $w^{Ch,2}_{Ph}$ satisfy conditions 1.-4. and condition

[TABLE]

Therefore, the preference parameters compatible with this piece of preference information are represented by the points in the trapezoid $W_{I\succsim G}$ having vertices $P_{1}\equiv(\frac{1}{2},\frac{1}{2})$, $P_{2}\equiv(\frac{1}{6},\frac{1}{2})$, $P_{4}\equiv(0,\frac{1}{3})$ and $O\equiv(0,0)$ in Figure 1. The centroids of $W_{G\succsim I}$ and $W_{I\succsim G}$ are the pairs of weights $(w^{Ch,2}_{M},w^{Ch,2}_{Ph})$ $B_{G\succsim I}\equiv(\frac{1}{18},\frac{4}{9})$, and $B_{I\succsim G}\equiv(\frac{1}{6},\frac{1}{3})$, respectively, that have been used as examples of parameters for which $U_{LDCh}(G)>U_{LDCh}(I)$ and $U_{LDCh}(G)<U_{LDCh}(I)$.

The above considerations show that a special attention has to be given to robustness concerns related to the application of the decision model. In general, it is always advisable to avoid to consider only one instance of the decision model compatible with the preference information provided by the DM (in our example the dean) because its choice is arbitrary to some extent. One possible answer to this type of situations is to distinguish preferences that hold for all compatible decision models and preferences that hold for at least one compatible decision model, called respectively necessary and possible preferences ([8, 9, 22]). With respect to students $G$, $H$ and $I$ we have to conclude that $G$ is necessarily preferred to $H$ as well as $I$ is necessarily preferred to $G$ in consequence of its dominance. Instead, with respect to students $G$ and $I$ we have only a possible preference both for $G$ over $I$ and for $I$ over $G$.

One can try to give a more precise evaluation of the range of compatible decision models for which student $G$ is preferred to student $I$ and vice versa by estimating the probability that picking randomly a vector of compatible parameters one gets $U_{LDCh}(G)>U_{LDCh}(I)$ or the opposite. Let us observe that the overall evaluations of students $G,H$ and $I$ involve only the weights $w^{Ch,2}_{M}$ and $w^{Ch,2}_{Ph}$. Consequently, to compare students $G$, $H$ and $I$, the set of the compatible decision models can be identified with the set $W$ of pairs of weights $\left(w^{Ch,2}_{M},w^{Ch,2}_{Ph}\right)$ satisfying conditions 1.-4. (in fact, only conditions 1. and 2., because conditions 3. and 4. are related to the weights $w^{Ch,1}_{M}$ and $w^{Ch,1}_{Ph}$). The set $W$ of compatible weights $\left(w^{Ch,2}_{M},w^{Ch,2}_{Ph}\right)$ are represented by the points in triangle having vertices $O$, $P_{1}$ and $P_{3}$ in Figure 1. We have already seen that the triangle $W_{G\succsim I}$, which vertices are $P_{2}$, $P_{3}$ and $P_{4}$, is composed of the set of pairs of weights $(w^{Ch,2}_{M},w^{Ch,2}_{Ph})$ for which $G$ is at least as good as $I$, while the trapezoid $W_{I\succsim G}$, which vertices are $O$, $P_{1}$, $P_{2}$ and $P_{4}$, is composed of the set of pairs of weights for which $I$ is at least as good as $G$. One can suppose a uniform distribution on the set of compatible pairs of weights $W$. Under this hypothesis, the probability of random picking a decision model for which $G$ is preferred to $I$ is given by the ratio between the area of $W_{G\succsim I}$ and the area of $W$, while the analogous probability for $I$ being preferred to $G$ is given by the ratio between the area of $W_{I\succsim G}$ and the area of $W$. Since the area of $W$, $W_{G\succsim I}$ and $W_{I\succsim G}$ are $\frac{1}{8}$, $\frac{1}{72}$ and $\frac{8}{72}$, respectively, we have that the probability of obtaining $G$ preferred over $I$ is $\frac{1}{9}$, while the probability of having $I$ preferred over $G$ is $\frac{8}{9}$. Instead, we have already seen that for all compatible decision models $G$ is preferred to $H$ as well as $I$ is preferred to $H$, so that the corresponding probabilities are equal to 1. One could also look for the probability for which each student $G$, $H$ and $I$ attains the first, the second and the third ranking position. Since we have already seen both $G$ and $I$ are always preferred to $H$, we have to conclude that $H$ takes always the third ranking position, so that his probability of being the third is 1 while the probability of being the first or the second is null. Instead $G$ takes the first ranking position if he is preferred to $I$, otherwise he is the second. Consequently, the probability that $G$ attains the first ranking position is $\frac{1}{9}$, while the probability of being the second is $\frac{8}{9}$. Analogously, we have that for $I$, the probabilities of being the first and the second are $\frac{8}{9}$ and $\frac{1}{9}$, respectively. For $G$ and $I$ the probability of being the third is null. Let us conclude this section by discussing an important element of the decision aiding procedure we presented in this example. It is strongly based on the reference points that are used to create the partition of the interval of evaluations. For instance, in our example, the dean is considering not so good notes, those not greater than 25 and good notes from 26 upwards. Of course these reference points have to be fixed together with the DM because they should express real aspects of his perception of evaluations on considered criteria. Overall, since a decision aiding procedure aims at constructing some arguments explaining the decision to be taken [41], a specific effort has to be put in avoiding to reference points that could permit to represent the preference expressed by the DM but that could be perceived by him as artificial and not credible. Instead, since the idea of reference point is so natural in the human perceptions and decisions (see e.g. [25, 38]) it is of fundamental importance for the successful application of the decision aiding procedure to discuss the reference points with the DM that has to accept them as meaningful and sensible.

In the following we shall show how to apply systematically the decision analysis approach proposed in this introductory example.

3 Basic concepts

In this section we shall briefly recall the Choquet integral (Section 3.1) and the level dependent Choquet integral (Section 3.2). A more detailed description of the two preference models can be found in [17] and [21], respectively.

3.1 The Choquet integral preference model

Let us consider a set of alternatives $A$ to be evaluated with respect to criteria $G=\{g_{1},\ldots,g_{n}\}$ where, for each $g_{i}\in G$ and for each $a\in A$, $g_{i}(a)\in\mbox{\rams\char 82\relax}$ represents the evaluation of $a$ on $g_{i}$. For the sake of simplicity and without loss of generality, we suppose that all criteria have an increasing direction of preference, that is, the greater $g_{i}(a)$, the better is $a$ on $g_{i}$. In the following, we shall denote criterion $g_{i}$ by its index $i$ only. Therefore, the statements $g_{i}\in G$ or $i\in G$ will be equivalent.

A capacity is a set function $\mu:2^{G}\rightarrow[0,1]$ such that $\mu(B)\leqslant\mu(E)$ for all $B\subseteq E\subseteq G$ (monotonicity constraints), $\mu(\emptyset)=0$ and $\mu(G)=1$ (normalization constraints). The Möbius transform of $\mu$, instead, is a set function $m:2^{G}\rightarrow\mbox{\rams\char 82\relax}$ such that, for all $E\subseteq G$, $\mu(E)=\sum_{B\subseteq E}m(B)$ [42]. Consequently, the monotonicity and normalization constraints defined above can be rewritten in terms of $m$ as follows:

$\forall i\in G$ and for all $E\subseteq G\setminus\{i\}$, $\displaystyle\sum_{R\subseteq E}m\left(R\cup\{i\}\right)\geqslant 0$ (monotonicity constraints),

• 2.

$m\left(\emptyset\right)=0$, $\displaystyle\sum_{E\subseteq G}m(E)=1$ (normalization constraints).

In this case, the Choquet integral of $\mathbf{x}=\left(x_{1},\ldots,x_{n}\right)\in\mbox{\rams\char 82\relax}^{n}$ w.r.t. $\mu$ can be written as

[TABLE]

where $(\cdot)$ stands for a permutation of the indices of criteria such that $0=x_{(0)}\leqslant\ldots\leqslant x_{(i)}\leq\ldots\leqslant x_{(n)}$ and $N_{i}=\{j\in G:x_{j}\geqslant x_{(i)}\}$ for all $i=1,\ldots,n.$ From now on, we shall use $a$ to denote an alternative belonging to $A$ and $\mathbf{a}$ to indicate the vector which components are the evaluations of $a$ on the considered criteria.

The use of (7) involves the knowledge of $2^{n}-2$ parameters (one for each subset of $G$ apart from $\emptyset$ and $G$ since $\mu(\emptyset)=0$ and $\mu(G)=1$). Anyway, for real world applications, $2$-additive capacities are enough to represent the preferences of the DM where a capacity is said $k$-additive iff its Möbius transform $m$ is such that $m(E)=0$ for all $E\subseteq G:|E|>k$ [16]. In this case, the Choquet integral of $\mathbf{x}$ can be rewritten as

[TABLE]

while monotonicity and normalization constraints become

$\displaystyle m\left(\emptyset\right)=0,\,\,\,\sum_{i\in G}m\left(\left\{i\right\}\right)+\sum_{\left\{i,j\right\}\subseteq G}m\left(\left\{i,j\right\}\right)=1,$ (normalization constraints)

• 2.

\left\{\begin{array}[]{l}\displaystyle m\left(\left\{i\right\}\right)\geq 0,\;\forall i\in G,\\[5.69054pt] \displaystyle m\left(\left\{i\right\}\right)+{\sum_{j\in E}}\>m\left(\left\{i,j\right\}\right)\geqslant 0,\;\forall i\in G\>\mbox{and}\>\forall\>E\subseteq G\setminus\left\{i\right\}, E\neq\emptyset\end{array}\right. (monotonicity constraints).

3.2 The level dependent Choquet integral

A generalized capacity $\mu^{L}$ is a function $\mu^{L}:2^{G}\times[\alpha,\beta]\rightarrow[0,1]$, with $[\alpha,\beta]\subseteq\mathbb{R}$, such that:

1.a) for all $t\in\left[\alpha,\beta\right]$ and for all $B\subseteq E\subseteq G,\;\mu^{L}(B,t)\leqslant\mu^{L}(E,t)$,

2.a) for all $t\in\left[\alpha,\beta\right]$, $\mu^{L}(\emptyset,t)=0$ and $\mu^{L}(G,t)=1$,

3.a) for all $t\in\left[\alpha,\beta\right]$ and for all $E\subseteq G,\;\mu^{L}(E,t)$ is Lebesgue measurable if considered as a function of $t$.

Let us observe that items 1.a) and 2.a) above imply that $\mu^{L}$ gives a capacity for each $t\in[\alpha,\beta]$.

The level dependent Choquet integral of $\mathbf{x}=[x_{1},\ldots,x_{n}]\in[\alpha,\beta]^{n}$ w.r.t. $\mu^{L}$, can be defined as follows:

[TABLE]

If the values of criteria are normalized between 0 and 1 and, consequently, $\mathbf{x}\in[0,1]^{n}$, we get

[TABLE]

3.2.1 Interval level dependent capacity

The use of the level dependent Choquet integral can be limited by noticing that it involves the definition of an arbitrary number of capacities $\mu^{L}(\cdot,t)$, one for each level $t\in[\alpha,\beta].$ To overcome this problem, in [21] the authors suggest the use of an interval level dependent capacity. A generalized capacity $\mu^{L}$ is called interval level dependent if there exists

a partition $a_{0},a_{1},\ldots,a_{p-1},a_{p}$, of the interval $[\alpha,\beta]$, where $\alpha=a_{0}<a_{1}<\ldots<a_{p-1}<a_{p}=\beta$,

• 2.

$p$ capacities $\mu_{1},\ldots,\mu_{p}$ defined on $2^{G}$,

such that, for all $E\subseteq G$, $\mu^{L}(E,t)=\mu_{r}(E)$ if $t\in[a_{r-1},a_{r}[$, $r=1,\ldots,p-1,$ and $\mu^{L}(E,t)=\mu_{p}(E)$ if $t\in[a_{p-1},a_{p}]$. For the sake of simplicity, from now on, even if we shall write $[a_{r-1},a_{r}[$ with $r=1,\ldots,p$, we shall consider as last interval the closed interval $[a_{p-1},a_{p}]$. Moreover, let us observe that in defining the partition we can also consider closed intervals $\left([a_{r-1},a_{r}]\right)$, half-opened intervals ($[a_{r-1},a_{r}[$ or $]a_{r-1},a_{r}]$) or opened intervals $\left(]a_{r-1},a_{r}[\right)$. What is important is that all subintervals of the decomposition are pairwise disjoint.

If $\mu^{L}$ is an interval level dependent capacity, the following theorem allows to compute the level dependent Choquet integral of $\mathbf{x}$ w.r.t. $\mu^{L}$ as the sum of $p$ Choquet integrals, one for each interval $[a_{r-1},a_{r}[,\;r=1,\ldots,p$:

Theorem 3.1**.**

*(see [21])

If $\mu^{L}$ is an interval level dependent capacity on $2^{G}$ relative to the breakpoints

$a_{0},a_{1},\ldots,a_{p-1},a_{p}\in[\alpha,\beta]$ and to the capacities $\mu_{1},\ldots,\mu_{p}$, then for each $\mathbf{x}\in[\alpha,\beta]^{n}$*

[TABLE]

where $\mathbf{x}^{r}\in[0,a_{r}-a_{r-1}]^{n}$ is the vector which components $x_{i}^{r}$ are defined as

[TABLE]

for all $i=1,\ldots,n.$

Consequently, the level dependent Choquet integral can be written as

[TABLE]

where ${(\cdot)}$ stands for a permutation of the indices of criteria so that $0=x^{r}_{(0)}\leqslant\ldots\leqslant x^{r}_{(i)}\leq\ldots\leqslant x^{r}_{(n)}$ and $N_{i}^{r}=\{j\in G:x^{r}_{j}\geqslant x^{r}_{(i)}\}$.

It is interesting to observe that a result similar to the one shown in Theorem 3.1 was proposed in [33] where the Choquet integral with respect to an interval level dependent capacity was formulated in terms of an ordinal sum of Choquet integrals based on capacities $\mu_{r}$. More precisely, on the basis of [32], consider

the above partition of $\left[\alpha,\beta\right]$ obtained with $\alpha=a_{0}<a_{1}<\ldots<a_{p-1}<a_{p}=\beta$,

• 2.

$p$ aggregation functions $A_{r},r=1,\ldots,p$, that are, functions $A_{r}:[a_{r-1},a_{r}]^{n}\rightarrow[a_{r-1},a_{r}]$ non decreasing in their arguments and such that $A_{r}(a_{r-1},\ldots,a_{r-1})=a_{r-1}$ and $A_{r}(a_{r},\ldots,a_{r})=a_{r}$,

• 3.

an increasing bijection $\varphi:[\alpha,\beta]\rightarrow\mathbb{R}.$

Then the $\varphi$-ordinal sum of aggregation functions $A_{1},\ldots,A_{p}$ is given by

[TABLE]

with $\tilde{x}^{r}_{i}=max\left(a_{r},min\left(a_{r-1},x_{i}\right)\right),i=1,\ldots,n$.

[33] proves that

[TABLE]

with $id(x)=x$ for all $x\in[\alpha,\beta]$ and $Ch^{r}({\mathbf{x},\mu_{r}})$ being the restriction of $Ch({\mathbf{x},\mu_{r}})$ to $[a_{r-1},a_{r}]^{n}$, that is,

[TABLE]

The formulation (10) has the merit of expressing the level dependent Choquet integral based on an interval level dependent capacity in terms of ordinal sum which is an important issue originally proposed by Birkhoff [5] for posets and lattices and after applied for $t$-norms and $t$-conorms [26] and copulas [36]. From a more operational point of view, formulations (8) and (11) permit to represent the level dependent Choquet integral based on an interval level dependent capacity in terms of usual Choquet integrals based on capacities $\mu_{r},r=1,\ldots,p$. In fact the addends of the sum in (8) and (11) are the same, that is

[TABLE]

because

[TABLE]

and in consequence of the translation invariance of the Choquet integral, that is,

[TABLE]

for all capacities $\mu$, all vectors $\mathbf{x}\in\mathbb{R}^{n}$ and all constants $c\in\mathbb{R}$ with $\mathbf{c}=[c,\ldots,c]\in\mathbb{R}^{n}$. Indeed, by (13) and (14) one can get

[TABLE]

and, consequently, (12).

3.2.2 Möbius transform of level dependent capacity

Given a level dependent capacity $\mu^{L}$, its Möbius transform is a function $m^{L}:2^{G}\times(\alpha,\beta)\rightarrow\mathbb{R}$, such that, for all $E\subseteq G$ and for all $t\in[\alpha,\beta]$

[TABLE]

In terms of Möbius transform, properties 1.a) and 2.a) are restated as:

1.b) $\displaystyle m^{L}(\emptyset,t)=0,\,\,\,\sum_{E\subseteq G}m^{L}(E,t)=1,\;\forall t\in[\alpha,\beta]$,

2.b) $\displaystyle\forall\>i\in G\;\mbox{and}\;\forall E\subseteq G\setminus\left\{i\right\},\;\sum_{R\subseteq E}\>m^{L}(R\cup\left\{i\right\},t)\geqslant 0,\;\forall t\in[\alpha,\beta]$.

Considering an interval level dependent capacity where $a_{0},a_{1},\ldots,a_{p}$ are the elements decomposing the interval $[\alpha,\beta]$ and $m_{r}$ the Möbius transform of the capacities $\mu_{r}$, $r=1,\ldots,p,$ the level dependent Choquet integral (8) can be written in the following way

[TABLE]

where the $x^{r}_{i},\;i=1,\ldots,n$ have been defined above.

3.2.3 $k$-additive interval level dependent capacity

To apply in a more manageable way the level dependent Choquet integral, similarly to what has been done for a capacity, $k$-order additive interval level dependent capacities can be defined. An interval level dependent capacity $\mu^{L}$ is said $k$-additive $(1\leqslant k\leqslant n-1)$ if, for all $t\in[\alpha,\beta]$, $m^{L}(E,t)=0$ for all $E\subseteq G$ such that $|E|>k$. Since, as observed above, giving a weight to each criterion and to each pair of criteria is enough to represent the preferences of the DM, 2-additive interval level dependent capacities are used in practice. The value that a $2$-additive interval level dependent capacity $\mu^{L}$ assigns to a set $E\subseteq G$ can be expressed in terms of its Möbius transform as follows: $\forall E\subseteq G\;\mbox{and}\;\forall t\in[\alpha,\beta]$

[TABLE]

With regard to 2-additive interval level dependent capacities, properties 1.b) and 2.b) can be restated as follows:

1.c)

$\displaystyle m^{L}\left(\emptyset,t\right)=0,\,\,\,\sum_{i\in G}m^{L}\left(\left\{i\right\},t\right)+\sum_{\left\{i,j\right\}\subseteq G}m^{L}\left(\left\{i,j\right\},t\right)=1,$

2.c)

\left\{\begin{array}[]{l}\displaystyle m^{L}\left(\left\{i\right\},t\right)\geqslant 0,\;\forall i\in G,\\[5.69054pt] \displaystyle m^{L}\left(\left\{i\right\},t\right)+{\sum_{j\in E}}\>m^{L}\left(\left\{i,j\right\},t\right)\geqslant 0,\;\forall i\in G\>\mbox{and}\>\forall\>E\subseteq G\setminus\left\{i\right\}, E\neq\emptyset.\end{array}\right.

Proposition 3.1**.**

Given

an interval level dependent capacity $\mu^{L}$ over $2^{G}\times[\alpha,\beta]$, with $a_{0},\ldots,a_{p}$ being a partition of the interval $[\alpha,\beta]$ and $\mu_{r}$ the corresponding capacities, $r=1,\ldots,p,$

• 2.

$m^{L}$* and $m_{r}$ the Möbius transforms of $\mu^{L}$ and $\mu_{r}$, respectively,*

then $\mu^{L}$ is a $k$-additive interval level dependent capacity if and only if all $\mu_{r}$ are $k$-additive, $r=1,\ldots,p$.

Proof.

It is a straightforward consequence of the definitions of $k$-additive capacities and $k$-additive interval level dependent capacities. ∎

Considering Proposition 3.1, if $\mu^{L}$ is $2$-additive, then the level dependent Choquet integral may be also reformulated as follows:

[TABLE]

where the $x_{i}^{r}$ , $i=1,...,n$, have been defined above.

3.2.4 Importance indices

Taking into account a capacity, the importance of a criterion $i$ is not dependent on itself only but also on its contribution to all possible coalitions of criteria. For this reason, the Shapley index, defining the importance of a criterion $i$, has been introduced in [43]. In case of a level dependent capacity, the importance of a criterion $i$ may (but has not to) vary when the level $t$ is changed. Consequently, given a generalized capacity $\mu^{L}$, we have that [21]:

[TABLE]

Considering the Möbius transform $m^{L}$ of $\mu^{L}$, the importance index $\phi(\mu^{L},i,t)$ can be rewritten as

[TABLE]

Finally, if $\mu^{L}$ is an interval level dependent capacity, then

[TABLE]

From (16)-(18), the comprehensive importance of criterion $i$ can be obtained as [21]:

[TABLE]

In particular, for an interval level dependent capacity, the importance of criterion $i$ on the interval $[a_{r-1},a_{r}[$ is given by

[TABLE]

and, consequently,

[TABLE]

For a $2$-additive interval level dependent capacity, the previous index can be rewritten as

[TABLE]

3.2.5 Interaction indices

With respect to the Choquet integral, interaction indices have been introduced by Murofushi and Soneda [35] considering only couples of criteria, and by Grabisch [16] with respect to all possible subsets of criteria.

Given a level dependent capacity $\mu^{L},\;E\subseteq G$ and $t\in[\alpha,\beta]$, the interaction index can be computed as [21]

[TABLE]

The interaction index $I(\mu^{L},E,t)$ can be represented in terms of Möbius transform as follows:

[TABLE]

If $\mu^{L}$ is an interval level dependent capacity, then

[TABLE]

As a consequence, the importance of criteria in $E$ for the interval $[a_{r-1},a_{r}[$ is

[TABLE]

and, therefore,

[TABLE]

If $E=\{i,j\}$ and $\mu^{L}$ is a 2-additive interval level dependent capacity, then (19) can be rewritten as

[TABLE]

4 NAROR and SMAA applied to the level dependent Choquet integral preference model

In this section we shall describe the extension of NAROR and SMAA to the level dependent Choquet integral.

4.1 Non Additive Robust Ordinal Regression (NAROR)

Every Multiple Criteria Decision Aiding (MCDA) method needs some parameters to be implemented [18]. In particular, when the parameters related to the DM’s preferences have to be set, it can be chosen a direct or an indirect technique to get them. In the first case, the DM provides all the values of the parameters requested by the method. In the second case, the DM provides only some statements which are subsequently used to obtain the parameters requested by the model.

The NAROR [3] belongs to the family of ROR methods (see [8, 9, 22]) and, as any ROR method, it is based on the indirect preference information.

Regarding this preference information, the DM can provide (among parenthesis the constraints translating the corresponding piece of preference information):

a possibly incomplete binary relation $\succsim^{DM}$ on a subset of alternatives $A^{*}\subseteq A$ for which $a\succsim^{DM}b$ iff $a$ is at least as good as $b$ $\left(Ch^{L}(\mathbf{a},\mu^{L})\geqslant Ch^{L}(\mathbf{b},\mu^{L})\right)$,

• 2.

a possibly incomplete binary relation $\succsim^{DM}$ on a subset of criteria $G^{*}\subseteq G$, for which $g_{i}\succsim^{DM}g_{j}$ iff $g_{i}$ is at least as important as $g_{j}$ $\left(\phi(\mu^{L},i)\geqslant\phi(\mu^{L},j)\right)$,

• 3.

a possibly incomplete binary relation $\succsim^{DM}_{[a_{r-1},a_{r}[}$ on a subset of criteria $G^{*}\subseteq G$, for which

$g_{i}\succsim_{[a_{r-1},a_{r}[}^{DM}g_{j}$ iff, considering the evaluations in the interval $[a_{r-1},a_{r}[$, $g_{i}$ is at least as important as $g_{j}$ $\left(\phi(\mu^{L},i,[a_{r-1},a_{r}[)\geqslant\phi(\mu^{L},j,[a_{r-1},a_{r}[)\right)$111On the basis of $\succsim^{DM}$, a strict preference relation $\succ^{DM}$ and an indifference relation $\sim^{DM}$ can be defined such that $a\succ^{DM}b$ iff $a\succsim^{DM}b$ and $not(b\succsim^{DM}a)$, while $a\sim^{DM}b$ iff $a\succsim^{DM}b$ and $b\succsim^{DM}a$. Analogously, on the basis of $\succsim_{[a_{r-1},a_{r}[}^{DM}$, the relations $\succ_{[a_{r-1},a_{r}[}^{DM}$ and $\sim_{[a_{r-1},a_{r}[}^{DM}$ can be defined. ,

• 4.

positive or negative interactions between criteria $g_{i}$ and $g_{j}$ in a comprehensive way ($I(\mu^{L},\{i,l\})>~{}0$ if $g_{i}$ and $g_{j}$ are positively interacting or $I(\mu^{L},\{i,j\})<0$ if $g_{i}$ and $g_{j}$ are negatively interacting) or considering a particular interval of evaluations $\left(I(\mu^{L},\{i,j\},[a_{r-1},a_{r}[)\right.>0$ if $g_{i}$ and $g_{j}$ are positively interacting or $I(\mu^{L},\{i,j\},[a_{r-1},a_{r}[)<0$ if $g_{i}$ and $g_{j}$ are negatively interacting).

Considering a 2-additive interval level dependent capacity $\mu^{L}$ relative to the breakpoints $a_{0},a_{1},\ldots,a_{p}$ and the Möbius transform $m_{r}$ of the capacities $\mu_{r}$, $r=1,\ldots,p$, to check if there exists at least one interval level dependent Choquet integral compatible with the preferences provided by the DM, the following LP problem has to be solved:

[TABLE]

If $E^{DM}$ is feasible and $\varepsilon^{*}>0$, then there exists at least one interval level dependent capacity compatible with the preferences provided by the DM, otherwise, there is not any interval level dependent capacity compatible and, therefore, the constraints causing this incompatibility can be identified by using one of the methods presented in [34].

In general, if there exists one interval level dependent capacity compatible with the preferences provided by the DM, there exists more than one and, therefore, the choice of only one of them can be considered arbitrary to some extent. For this reason, ROR takes into account all of them defining a necessary $\succsim^{N}$ and a possible $\succsim^{P}$ preference relation. The necessary preference relation holds between two alternatives $a$ and $b$ if $a$ is at least as good as $b$ for all compatible models, while the possible preference relation holds between $a$ and $b$ if $a$ is at least as good as $b$ for at least one compatible model.

From a computational point of view, the two preference relations can be computed as follows:

$a\succsim^{N}b$ iff $E^{N}(a,b)$ is infeasible or $\varepsilon^{N}\leq 0$ where $\varepsilon^{N}=\max\varepsilon$ subject to $E^{N}(a,b)$ and

[TABLE]

• 2.

$a\succsim^{P}b$ iff $E^{P}(a,b)$ is feasible and $\varepsilon^{P}>0$ where $\varepsilon^{P}=\max\varepsilon$ subject to $E^{P}(a,b)$ and

[TABLE]

4.2 Stochastic Multicriteria Acceptability Analysis (SMAA)

SMAA  [28, 29] is a family of MCDA methods which take into account uncertainty or imprecision on the evaluations and preference model parameters. In this section we describe SMAA-2  [29], since its underlying preference model is a value function and the Choquet integral preference model belongs to this family. In order to apply SMAA to the level dependent Choquet integral, we shall denote by $\chi$ the evaluation space and by ${\cal M}$ the set of all interval level dependent capacities satisfying the constraints in $E^{DM}$.

As ROR methods, SMAA methods take into account all the models compatible with the preferences provided by the DM even if in a different way. The indirect preference information is composed of two probability distributions, $f_{\chi}$ and $f_{{\cal M}}$, defined on $\chi$ and ${\cal M}$, respectively.

Given $\xi\in\chi$ and $\mu^{L}\in{\cal M}$, SMAA methods define the rank function

[TABLE]

(where $\rho(false)=0$ and $\rho(true)=1$ and $\xi_{a}$ are the evaluations of $a$ in the matrix $\xi$) that gives the rank position of alternative $a$. On the basis of this rank function, SMAA-2 computes the set of interval level dependent capacities ${\cal M}^{s}(a,\xi)\subseteq{\cal M}$ for which alternative $a$ assumes rank $s=1,2,\ldots,|A|$, as follows:

[TABLE]

The following indices are therefore computed in SMAA-2:

The rank acceptability index that measures the variety of different parameters compatible with the DM’s preference information giving to $a$ the rank $s$:

[TABLE]

$b^{s}(a)$ gives the probability that $a$ has rank $s$, and it is within the range $[0,1]$;

• 2.

The pairwise winning index [31] that is defined as the frequency with which an alternative $a$ is preferred to an alternative $c$ in the space of the interval level dependent capacities:

[TABLE]

From a computational point of view, the multidimensional integrals defining the considered indices are estimated by using the Monte Carlo method. In particular, since the preference information provided by the DM and translated into constraints in $E^{DM}$ defines a convex space of parameters (interval level dependent capacities), the Hit-And-Run (HAR) method can be used to sample several of these sets of parameters [44, 45, 49]. The application of the level dependent Choquet integral to each of these sets of parameters produces a ranking of the alternatives at hand. Consequently, considering all these rankings simultaneously, the rank acceptability index, as well as the pairwise winning indices described above, can be computed for each alternative and for each pair of alternatives, respectively.

Let us conclude this section by observing that the proposed method could be also used to explain a full ranking provided by the DM and taking into account some preferences on the importance and the interaction between criteria he wishes to express. While, in general, the information provided by the DM regards a set composed of few alternatives and on the basis of this preference information we aim to give recommendations on the whole set of alternatives at hand, in some cases the DM has a clear idea of the ranking of the alternatives and he wishes “only” to justify them. Technically, his preferences are represented by the importance assigned to the criteria and to the possible interactions between criteria. From a formal point of view, the full ranking of the alternatives given by the DM is translated into inequality constraints between consecutive alternatives in the ranking as described in Section 4.1. For example, if the alternatives at hand are denoted by $a,b,c,d$ and the DM is convinced that they are ranked in this order, then it is necessary to add the following three inequalities: $Ch^{L}(\textbf{a},\mu^{L})\geqslant Ch^{L}(\textbf{b},\mu^{L})+\varepsilon$, $Ch^{L}(\textbf{b},\mu^{L})\geqslant Ch^{L}(\textbf{c},\mu^{L})+\varepsilon$ and $Ch^{L}(\textbf{c},\mu^{L})\geqslant Ch^{L}(\textbf{d},\mu^{L})+\varepsilon$. If there is not any set of parameters compatible with this full preference, then, this means that it is inconsistent. One can therefore check for the cause of the inconsistency as explained in Section 4.1. If, instead, there is at least one set of parameters for which the alternatives’ ranking is restored, then there could exist more than one. The application of NAROR and SMAA in this situation is meaningless. Indeed, since all vectors of parameters provide exactly the same ranking (being the one provided by the DM on the whole set of alternatives) the necessary preference relation will give back a total order of the alternatives being the same total order of alternatives the DM already gave. Each alternative $a$ will get $b^{s}(a)=100\%$ in correspondence of the the position $s$ it filled in the ranking given by the DM and, finally, one between $p(a,c)$ and $p(c,a)$ is equal to 100%, depending on the fact that the DM stated that $a$ is preferred to $c$ or that $c$ is preferred to $a$. Therefore, we could sample a set of vectors of parameters compatible with the preferences provided by the DM and, then, computing its barycenter obtained by averaging, component by component, all sampled vectors. On the basis of the values in the barycenter, one can compute the Shapley indices and the interaction indices giving, therefore, information on the ranking of the criteria with respect to their importance as well as on the possible positive or negative interactions between criteria that could vary from one level of evaluations to the other. Let us observe that the ranking provided by the DM could admit some ex-aequo between the alternatives filling the same position. In this case, the weak preference of $a$ over $c$ is translated into the constraint $Ch^{L}(\textbf{a},\mu^{L})\geqslant Ch^{L}(\textbf{c},\mu^{L})$ (without including $\varepsilon$) and the two pairwise winning indices $p(a,c)$ and $p(c,a)$ would be equal to zero.

5 Ordinal regression considering level dependent capacities changing with continuity

Until now we have considered interval level dependent capacities, that is, level dependent capacities that change with a jump from one interval of evaluations to another but remaining constant within each single interval. Let us observe that this could be an acceptable working hypothesis for real life decision problems in which importance and interaction of criteria change from one interval of evaluations to another. However, it is also reasonable assuming that the importance and the interaction between criteria and, consequently, the level dependent capacities, change with continuity inside each interval of evaluations of considered criteria. Going more in depth in this case, in the following we present a simple and manageable procedure permitting to elicit a level dependent capacity $\mu^{L}(\cdot,t)$ changing with continuity with respect to $t$. The continuous level dependent capacities that we consider are related again to the partition of the interval $[\alpha,\beta]$ based on the finite sequence $\alpha=a_{0},a_{1},\ldots,a_{p-1},a_{p}=\beta$, that is, the same partition we have considered for the interval level dependent capacities. More precisely, the new continuous level dependent capacity will be defined once the $p+1$ capacities $\mu^{L}(\cdot,a_{0}),\mu^{L}(\cdot,a_{1}),\ldots,\mu^{L}(\cdot,a_{p})$ are known. These capacities can be induced, for example, from the preference information provided by the DM. For all $E\subseteq G$ and for all $t\in[\alpha,\beta]\setminus\{a_{0},a_{1},\ldots,a_{p-1},a_{p}\}$ we compute $\mu^{L}(E,t)$ as a linear interpolation between $\mu^{L}(E,a_{\underline{r}(t)})$ and $\mu^{L}(E,a_{\underline{r}(t)+1})$, where $\underline{r}(t)=max\{r\in\{0,1,\ldots,p\}:a_{r}<t\}$, as follows:

[TABLE]

From the equation above we obtain, therefore,

[TABLE]

Consequently, the level dependent Choquet integral can be formulated in terms of the $p+1$ capacities $\mu^{L}(\cdot,a_{0}),\mu^{L}(\cdot,a_{1}),\ldots,\mu^{L}(\cdot,a_{p})$ as follows. Reminding from [21] that

[TABLE]

and observing that, for all $t\in[x_{(i-1)},x_{(i)}[$,

[TABLE]

we can write

[TABLE]

The values of $\int_{x_{(i-1)}}^{x_{(i)}}\mu^{L}(N_{i}(\mathbf{x}),t)dt,$ $i=1,\ldots,n$, are computed considering the following two cases:

$\underline{r}(x_{(i-1)})=\underline{r}(x_{(i)})$: in this case, $x_{(i-1)}$ and $x_{(i)}$ belong to the same interval in the partition of $[\alpha,\beta]$. Therefore, $\int_{x_{(i-1)}}^{x_{(i)}}\mu^{L}(N_{i}(\mathbf{x}),t)dt$ is given by the area of the trapezoid $T$ in Figure 2, that is,

[TABLE]

By (21) and (22), $\int_{x_{(i-1)}}^{x_{(i)}}\mu^{L}(N_{i}(\mathbf{x}),t)dt$ can be expressed in terms of capacities $\mu^{L}(\cdot,a_{0}),\ldots,\mu^{L}(\cdot,a_{p})$, as follows:

[TABLE]

• 2.

$\underline{r}(x_{(i-1)})<\underline{r}(x_{(i)})$: in this case, $x_{(i-1)}$ and $x_{(i)}$ do not belong to the same interval in the partition of $\left[\alpha,\beta\right]$. Therefore, the value $\int_{x_{(i-1)}}^{x_{(i)}}\mu^{L}(N_{i}(\mathbf{x}),t)dt$ is given by the sum of area of trapezoids $T_{\underline{r}(x_{i-1})},T_{\underline{r}(x_{i-1})+1},\ldots,T_{\underline{r}(x_{i})}$, in Figure 3:

[TABLE]

[TABLE]

In this case, by (21), $\int_{x_{(i-1)}}^{x_{(i)}}\mu^{L}(\{g_{i}\in G:x_{i}\geqslant t\},t)dt$ can be expressed in terms of capacities $\mu^{L}(\cdot,a_{0}),\ldots,\mu^{L}(\cdot,a_{p})$, as follows:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

With respect to preference information in terms of importance and interaction between criteria, we can observe that on the basis of (21) for all $g_{i}\in G,$ and for all $t\in[\alpha,\beta]$, the importance index $\phi(i,\mu^{L},t)$ and the interaction index $I(E,\mu^{L},t)$ can be expressed as linear functions of the same capacities and, consequently, of the related Möbius transforms. Since, by (21), $\mu^{L}(E,t)$ can be expressed as linear interpolation of $\mu^{L}(E,a_{\underline{r}(t)})$ and $\mu^{L}(E,a_{\underline{r}(t)+1})$, $\phi(i,\mu^{L},t)$ can be expressed as linear interpolation of $\phi(i,\mu^{L},a_{\underline{r}(t)})$ and $\phi(i,\mu^{L},a_{\underline{r}(t)+1})$ and $I(E,\mu^{L},t)$ can be expressed as linear interpolation of $I(E,\mu^{L},a_{\underline{r}(t)})$ and $I(E,\mu^{L},a_{\underline{r}(t)+1})$. Indeed,

[TABLE]

[TABLE]

and

[TABLE]

[TABLE]

On the basis of this observation, also preference information in terms of importance and interaction of criteria can be represented as linear constraints on the unknowns $m^{L}(E,a_{0}),\ldots,m^{L}(E,a_{p})$ for all $E\subseteq G$. For fixing the ideas, let us consider the preference information related to the greater importance of criterion $g_{i}$ over criterion $g_{j}$ in the interval of evaluations $[\gamma,\delta]\subseteq[\alpha,\beta]$, denoted by $g_{i}\succ^{DM}_{[\gamma,\delta]}g_{j}$. It can be interpreted as

[TABLE]

that is equivalent to the following constraints

[TABLE]

Indeed, condition (26) is clearly necessary for (25), because it requires that the inequality $\phi(i,\mu^{L},t)>\phi(j,\mu^{L},t)$ holds for a subset of points in $[\gamma,\delta]$. (26) is also sufficient because independently on the interval among $[\gamma,a_{\underline{r}(\gamma)+1}],[a_{\underline{r}(\gamma)+1},a_{\underline{r}(\gamma)+2}],\ldots,[a_{\underline{r}(\delta)-1},a_{\underline{r}(\delta)}]$ and $[a_{\underline{r}(\delta)},\delta]$ to which $t$ belongs, $\phi(i,\mu^{L},t)$ and $\phi(j,\mu^{L},t)$ are convex combinations with the same multipliers of the extremes of the interval. Consequently, if $\phi(i,\mu^{L},t)$ is greater than $\phi(j,\mu^{L},t)$ in the two extremes of the interval, then $\phi(i,\mu^{L},t)$ is greater than $\phi(j,\mu^{L},t)$ for all the $t$ in the same interval.

Reasoning in the same way, we can say that the preference information related to the positive (negative) interaction between criteria $g_{i}$ and $g_{j}$ in the interval of evaluations $[\gamma,\delta]\subseteq[\alpha,\beta]$ can be interpreted as

[TABLE]

that is equivalent to the constraint

[TABLE]

Let us observe that using (23) and (24), the level dependent Choquet integral ${Ch}^{L}(\mathbf{x},\mu^{L})$ can be expressed as a linear function of values $\mu^{L}(E,a_{0}),\ldots,\mu^{L}(E,a_{p})$ for all $E\subseteq G$ that, in turn, can be expressed in terms of the corresponding Möbius transforms $m^{L}(E,a_{0}),\ldots,m^{L}(E,a_{p})$. Therefore, preference information expressed in terms of pairwise preference comparisons ($a\succ^{DM}b$ or $a\sim^{DM}b$) can be translated into linear constraints in the unknowns $m^{L}(E,a_{0}),\ldots,m^{L}(E,a_{p})$ for all $E\subseteq G$. Consequently, results of ROR and SMAA can be obtained by solving LP problems and applying HAR algorithm as explained in Section 4 for the case of interval level dependent capacities.

5.1 Introductory example: continuous case

Let us continue the example introduced in Section 2 with the aim of illustrating the use of piecewise linear level dependent capacities in the perspective of tempered parsimony for MCDA models we are promoting. For this reason, also in this case, the example is presented quite analytically. The dean maintains the convictions already expressed on the importance and interaction between Mathematics and Physics. However, he thinks to present the evaluations of students to the school committee using a formal model that avoids jumps of importance and interaction between criteria in the interval of feasible notes. Consequently:

for notes smaller than 25, Physics is more important than Mathematics and there is a synergy between the two subjects,

• 2.

for notes greater than 25, Mathematics is more important than Physics and there is a redundancy between the two subjects.

With this aim, the dean considered the continuous level dependent capacity presented in the previous Section 5. More in detail, the interval $[18,30]$, where the admissible values for the notes lay, was split in the two subintervals $[18,25[$ and $]25,18]$ plus the singleton $\{25\}$, so that the whole level dependent capacity $\mu^{L}(E,t):2^{\{Mathematics,Physics\}}\times[18,30]\rightarrow[0,1]$ remains fixed once $\mu^{L}(E,18),\mu^{L}(E,25)$ and $\mu^{L}(E,30)$ are determined, with $E\in 2^{\{Mathematics,Physics\}}$. Let us remember that, in the following equations, we shall write, $M$ and $Ph$ instead of $Mathematics$ and $Physics$. Moreover, to simplify the notation, we shall write $\mu^{L}(M,t)$ and $\mu^{L}(Ph,t)$ instead of $\mu^{L}(\{M\},t)$ and $\mu^{L}(\{Ph\},t)$.

Let us observe that:

for notes in the interval $[18,25[$ Mathematics is more important than Physics, that is, $\mu^{L}(M,t)>\mu^{L}(Ph,t)$ for all $t\in[18,25[$,

• 2.

for notes in the interval $]25,30[$ Physics is more important than Mathematics, that is, $\mu^{L}(Ph,t)>\mu^{L}(M,t)$ for all $t\in]25,30]$,

• 3.

$\mu^{L}(M,t)$ and $\mu^{L}(Ph,t)$ have to be continuous on the whole interval $[18,30]$.

Taking into account the previous information, one has to conclude that

[TABLE]

Analogously, let us observe that:

for notes in the interval $[18,25[$ there is a redundancy between Mathematics and Physics, that is,

[TABLE]

• 2.

for notes in the interval $]25,30[$ there is a synergy between notes on Mathematics and Physics, that is,

[TABLE]

• 3.

$\mu^{L}\left(\{M,Ph\},t\right)$ has to be continuous on the whole interval $[18,30]$.

On the basis of the previous considerations, one has to conclude that

[TABLE]

so that, observing that $\mu^{L}(\{M,Ph\},t)=1$ for all $t\in[18,30]$, one has

[TABLE]

From (28) and (29), we therefore get

[TABLE]

Using the continuous level dependent capacity $\mu^{L}$, according to (24), the overall evaluations of students from $A$ to $I$ can be expressed as follows:

${Ch}^{L}(A,\mu^{L})=28;$

• 2.

${Ch}^{L}(B,\mu^{L})=26+\mu^{L}(M,25)\frac{(2\times 30-30-26)(30-26)}{2(30-25)}+\mu^{L}(M,30)\frac{(30+26-2\times 25)(30-26)}{2(30-25)}=$

[TABLE]

so that, reminding (30),

[TABLE]

• 3.

${Ch}^{L}(C,\mu^{L})=26+\mu^{L}(Ph,25)\frac{(2\times 30-30-26)(30-26)}{2(30-25)}+\mu^{L}(Ph,30)\frac{(30+26-2\times 25)(30-26)}{2(30-25)}=$

[TABLE]

so that, reminding (30),

[TABLE]

• 4.

${Ch}^{L}(D,\mu^{L})=23;$

• 5.

${Ch}^{L}(E,\mu^{L})=21+\mu^{L}(M,18)\frac{(2\times 25-25-21)(25-21)}{2(25-18)}+\mu^{L}(M,25)\frac{(25+21-2\times 18)(25-21)}{2(25-18)}=$

[TABLE]

so that, reminding (30),

[TABLE]

• 6.

${Ch}^{L}(F,\mu^{L})=21+\mu^{L}(Ph,18)\frac{(2\times 25-25-21)(25-21)}{2(25-18)}+\mu^{L}(Ph,25)\frac{(25+21-2\times 18)(25-21)}{2(25-18)}=$

[TABLE]

so that, reminding (30),

[TABLE]

• 7.

${Ch}^{L}(G,\mu^{L})=26+\mu^{L}(Ph,25)\frac{(2\times 30-29-26)(29-26)}{2(30-25)}+\mu^{L}(Ph,30)\frac{(29+26-2\times 25)(29-26)}{2(30-25)}=$

[TABLE]

so that, reminding (30),

[TABLE]

• 8.

${Ch}^{L}(H,\mu^{L})=26+\mu^{L}(M,25)\frac{(2\times 30-29-26)(29-26)}{2(30-25)}+\mu^{L}(M,30)\frac{(29+26-2\times 25)(29-26)}{2(30-25)}=$

[TABLE]

so that, reminding (30),

[TABLE]

• 9.

${Ch}^{L}(I,\mu^{L})=27+\mu^{L}(M,25)\frac{(2\times 30-30-27)(30-27)}{2(30-25)}+\mu^{L}(M,30)\frac{(30+27-2\times 25)(30-27)}{2(30-25)}=$

[TABLE]

so that, reminding (30),

[TABLE]

In consequence, the preferences expressed by the dean are translated into the following constraints in terms of continuous level dependent capacity $\mu^{L}$:

[TABLE]

that is,

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Observe that these inequalities obtained from the preferences expressed by the dean are in agreement with his considerations about the importance and the interaction of criteria. Indeed, from (31)-(34) we get what follows:

by (21) and reminding (30), for all $t\in]25,30]$ we obtain

[TABLE]

and

[TABLE]

so that, by (32), we get

[TABLE]

confirming the greater importance of Physics over Mathematics for notes greater than 25;

• 2.

putting together (31) and (35), we obtain

[TABLE]

so that, considering also (37) and reminding that $\mu^{L}(\{M,Ph\},t)=1$ for all $t$, we get

[TABLE]

confirming the synergy between Mathematics and Physics for notes greater than 25;

• 3.

by (21) and (30), for all $t\in[18,25[$ we obtain

[TABLE]

and

[TABLE]

so that, by (33), we get

[TABLE]

confirming the greater importance of Mathematics over Physics for notes smaller than 25;

• 4.

putting together (34) and (38), we obtain

[TABLE]

so that, considering also (39) and reminding that $\mu^{L}(\{M,Ph\},t)=1$ for all admissible $t$, we get

[TABLE]

confirming the redundancy between Mathematics and Physics for notes smaller than 25.

Since the preferences expressed by the dean on the students are in agreement with the importance and interaction between criteria we wanted to take into consideration, the compatible level dependent capacities $\mu^{L}$ are those satisfying conditions (30) and (31)-(34).

Now let us compare between them students $G,H$ and $I$ through the Choquet integral based on the piecewise linear compatible capacities $\mu^{L}$ compatible with the preferences expressed by the dean. Let us observe that the three students have notes greater than 25 both on Mathematics and Physics, so that the different evaluations provided by the Choquet integral depend only on the values $\mu^{L}(E,t),E\subseteq\{M,Ph\}$ and $t\in]25,30]$. Regarding the piecewise linear compatible capacities, they are determined only by the values of $\mu^{L}(M,30)$ and $\mu^{L}(Ph,30)$. Indeed, $\mu^{L}(\emptyset,t)=0$ and $\mu^{L}(\{M,Ph\},t)=1$ for all admissible value of $t$, while according to (35) and (36), for all $t\in]25,30]$, the values of $\mu^{L}(M,t)$ and $\mu^{L}(Ph,t)$ depend only on $\mu^{L}(M,30)$ and $\mu^{L}(Ph,30)$. Therefore, with respect to students $G,H$ and $I$, taking into account preferences expressed by the dean and translated into the constraints (32) and (36), the set of all compatible piecewise linear capacities are represented by the points inside the triangle which vertices are $O=(0,0),P_{1}=(0,0.5)$ and $P_{3}=(0.5,0.5)$ in the $\mu^{L}(M,t)-\mu^{L}(Ph,t)$ plane in Figure 4.

On the basis of the above remarks, comparing students $G,H$ and $I$, we have that:

as in the case of interval of level dependent capacity (and, in fact, as in the case of any multiple criteria aggregation procedure) the dominance of $I$ over $H$ ensures that $I$ is preferred to $H$ for any compatible piecewise linear dependent capacity, that is, there is a necessary preference of $I$ over $H$;

• 2.

since $G\succ H$ is equivalent to

[TABLE]

that is,

[TABLE]

by (32), we can conclude that $G$ is preferred to $H$ for all compatible piecewise linear dependent capacities, so that there is a necessary preference of $G$ over $H$;

• 3.

since $G\succ I$ and $I\succ G$ are equivalent to

[TABLE]

that is,

[TABLE]

there are compatible piecewise linear level dependent capacities $\mu^{L}$ for which $G$ is preferred to $I$ (for example, considering $\mu^{L}$ such that $\mu^{L}(M,30)=\frac{1}{126}$ and $\mu^{L}(Ph,30)=\frac{22}{45}$ representing point $B_{G\succsim I}$ in Figure 4 for which ${Ch}^{L}(G,\mu^{L})=27.4833$ and ${Ch}^{L}(I,\mu^{L})=27.4667$) and compatible piecewise linear level dependent capacities $\mu^{L}$ for which $I$ is preferred to $G$ (for example, considering $\mu^{L}$ such that $\mu^{L}(M,30)=\frac{11}{84}$ and $\mu^{L}(Ph,30)=\frac{11}{30}$ representing point $B_{I\succsim G}$ in Figure 4 for which ${Ch}^{L}(G,\mu^{L})=26.9464$ and ${Ch}^{L}(I,\mu^{L})=28.22$). Therefore, there is a a possible preference of $G$ over $I$ as well as a possible preference of $I$ over $G$ and there is no necessary preference between these two students.

More precisely, with respect to the comparison of students $G$ and $I$, we have that:

$G$ is preferred over $I$ for the piecewise linear level dependent capacities $\mu^{L}$ represented by the points in the triangle of vertices $P_{1}\equiv\left(0,\frac{1}{2}\right),P_{2}\equiv(\frac{1}{42},\frac{1}{2})$ and $P_{5}\equiv(0,\frac{7}{15})$ (the above mentioned point $B_{G\succsim I}$ is the barycentre of this triangle),

• 2.

$I$ is preferred over $G$ for the piecewise linear level dependent capacities $\mu^{L}$ represented by the points in the polygon of vertices $O\equiv(0,0),P_{3}\equiv\left(\frac{1}{2},\frac{1}{2}\right),P_{2}\equiv\left(\frac{1}{42},\frac{1}{2}\right)$ and $P_{5}\equiv\left(0,\frac{7}{15}\right)$ (the above mentioned point $B_{I\succsim G}$ is the barycentre of this polygon).

Consequently the probability that taking randomly a compatible piecewise linear level dependent capacity $\mu^{L}$, $G$ is preferred over $I$ is 0.3175%, corresponding to the ratio between the area of the triangle of vertices $P_{1},P_{2}$ and $P_{5}$, and the triangle of vertices $O,P_{1},P_{3}$, while the probability that $I$ is preferred over $G$ is 99.6825%, corresponding to the the ratio between the area of the polygon of vertices $O,P_{3},P_{2}$ and $P_{5}$, and the triangle of vertices $O$, $P_{1}$ and $P_{3}$. Considering the probability of being the first, the second or the third taking randomly a compatible piecewise linear level dependent capacity $\mu^{L}$, we have that:

as $G$ and $I$ are preferred to $H$ for all compatible $\mu^{L}$, there is a probability of 100% that $H$ is the third,

• 2.

as $I$ is the first when he is preferred to $G$ and the second otherwise, there is a probability of 0.3175% that $H$ is the first and a probability of 99.6825% that he is the second,

• 3.

finally, $G$ is the first with a probability of 99.6825% and the second with a probability of 0.3175%.

Let us observe that the considerations on the participation of the DM to the fixing of the reference points that we did at the end of Section 2 for the interval level dependent capacities hold also for the piecewise linear level dependent capacities. Let us also point out that, in agreement with what observed at the end of the previous subsection, we have handled the application of the Choquet integral with respect to piecewise linear level dependent capacity using linear constraints only. This confirms that a systematic application of ROR and SMAA to this model can be dealt by solving LP problems and using the HAR algorithm.

6 Illustrative example

In this section, we shall apply our methodology to a real world problem attracting the interest of experts and public opinion in the last years, that is the university rankings. As acknowledged in [11], nowadays there are plenty of university rankings, starting from the most known one, that is the Shanghai ranking (ARWU) [4], to the ranking provided by The Times [46]. All of them are based on different aggregation methods and different importance assigned to the criteria taken into account in the considered analysis.

In this case, we will concentrate our attention on the comparison of economics universities all over the world evaluated with respect to the teaching and learning (TL) aspect of their bachelor and master courses. The evaluations of the universities are measured by means of 4 subcriteria being Bachelor Graduation Rate (BGR), Masters Graduation Rate (MGR), Bachelors Graduation on Time (BGT) and Masters Graduation on Time (MGT). The description of the four criteria is given in Table 3.

The database used in this study is U-Multirank [48] where the evaluations can vary between 1 (Weak) to 5 (Very Good). For the sake of simplicity we are supposing that the ordinal evaluations from 1 to 5 are accepted by the DM as the corresponding utility value (for two procedures to assess utility values see [2, 6]). At the beginning, 1609 universities are considered. Removing those having unavailable data for at least one of the four criteria and those for which the considered aspect was not applicable, 501 universities were left. Finally, we considered only universities that are not presenting the same profile. In this way, the considered universities are 124. The methodology has been applied to the whole set of universities and the final results can be consulted clicking on the following link: supplementary results. Anyway, for space reasons we show here the results of the application performed on a subset of universities that have been selected as explained in the following. Since the most interesting comparison regards the one performed between non-dominated universities, following the same procedure used in the celebrated evolutionary algorithm NSGA-II [12], we ordered the universities in non-dominated fronts. This means that we put in the first front the non-dominated universities. After removing them, we put in the second front the non-dominated universities among the remaining ones, and so on. Finally, 13 different fronts have been built. To show the application of the method, we have chosen the biggest one, that is the 7th, composed of 19 different universities which performances on the four criteria are shown in Table 5(a). Let us observe that another front or other universities (for example those belonging to the same Country) could be chosen. Close to these performances, we report also the performances of few universities the DM knows well and on which he provided some preference information. For brevity, the universities have been labeled by $U_{h}$. Anyway, in the file reporting the results of the application of the proposed method to the whole set of universities, we denoted the universities with their name.

At first, the DM provided some preferences between criteria as well as some possible positive and negative interactions between pairs of criteria. In particular, he considered as benchmark “not so good universities”, being those with performances varying between 1 and 3 and “good universities” being those with performances between 4 and 5. In this way, the interval scale $[1,5]$ is split in two subintervals being $[1,3]$ and $]3,5]$. The DM expressed the willingness to focus the attention on the bachelor students in case of not so good evaluations on considered criteria, while he wants to give more importance to master students in evaluating universities presenting good performances. This perspective is represented in formal terms as follows:

in evaluating universities with not so good evaluations on considered criteria, $BGR$ is more important than $MGR$, while the vice versa is true in evaluating universities presenting good performances. This preference information is translated into the following constraints:

• (a)

$\phi\left(\mu^{L},\{BGR\},[1,3]\right)>\phi\left(\mu^{L},\{MGR\},[1,3]\right)$,

• (b)

$\phi\left(\mu^{L},\{BGR\},]3,5]\right)<\phi\left(\mu^{L},\{MGR\},]3,5]\right)$;

• 2.

in evaluating universities with not so good evaluations, $BGT$ is more important than $MGT$, while the viceversa is true in evaluating universities presenting good performances. This preference information is translated into the following constraints:

• (a)

$\phi\left(\mu^{L},\{BGT\},[1,3]\right)>\phi\left(\mu^{L},\{MGT\},[1,3]\right)$,

• (b)

$\phi\left(\mu^{L},\{BGT\},]3,5]\right)<\phi\left(\mu^{L},\{MGT\},]3,5]\right)$;

• 3.

$BGR$ and $BGT$ are positively interacting as well as $MGR$ and $MGT$ in evaluating universities with not so good performances on considered criteria. The same pairs of criteria are negatively interacting in evaluating universities with good performances. On one hand, the DM would like to give a bonus to not so good universities in which students (both bachelor and master) are able not only to complete their programmes but also to finish their studies in the expected time. On the other hand, since good universities are, in general, attended by good students, it is not so surprising that students would complete their programmes and would be also able to finish their studies in the expected time. For this reason, in this case, the considered pairs of criteria are negatively interacting. This piece of preference information is translated into the following constraints:

• (a)

$I\left(\mu^{L},\{BGR,BGT\},[1,3]\right)>0$,

• (b)

$I\left(\mu^{L},\{BGR,BGT\},]3,5]\right)<0$,

• (c)

$I\left(\mu^{L},\{MGR,MGT\},[1,3]\right)>0$,

• (d)

$I\left(\mu^{L},\{MGR,MGT\},]3,5]\right)<0$.

The DM provided also the following preference information on the universities which performances are shown in Table 5(b):

[TABLE]

To represent this preference information and, in particular, the fact that the importance of criteria as well as the interactions between criteria are dependent on the considered performances we have to use the level dependent Choquet integral. As already explained above, we consider the partition $a_{0}=1,$ $a_{2}=3,$ $a_{2}=5$ of the interval $[1,5]$. This means that we are taking into account two capacities, one on the interval $[1,3]$ and one on the interval $]3,5]$. Considering this partition, we solve the LP problem (20) presented in Section 4.1, obtaining $\varepsilon^{*}=0.25>0$. This means that the level dependent Choquet integral, with the considered partition, is able to represent the preferences provided by the DM.

Since the universities at hand belong to the same non-dominated front, the dominance relation is empty and, consequently, looking only at their performances nothing can be said. Consequently, as following step, we decided to apply the NAROR to check if the necessary and possible preference relations can give a more clear view of how the universities can be compared each other. Unlikely, the necessary preference relation is empty (apart from the trivial relations $U_{h}\succsim U_{h}$ for all $U_{h}$). This means that there is not a certain recommendation on the problem at hand since there are models compatible with the DM’s preferences for which each university can be preferred to any other and vice versa. For such a reason, as described in Section 4.2, we apply the SMAA methodology obtaining probabilistic preferences. At first, we computed the rank acceptability indices of each university.

Looking at Table 5, one can see that all but two universities can reach the first position but only three of them have a first rank acceptability index greater than 10%, that is, in the order of their rank acceptability index for the first position, $U_{16}$ ($b^{1}(U_{16})=60.842\%$), $U_{5}$ ($b^{1}(U_{5})=11.409\%$) and $U_{11}$ ($b^{1}(U_{11})=10.098\%$). Considering the tail of this ranking, only six universities cannot be in the last position. Moreover, the universities being most frequently the last are $U_{10}$ ($b^{19}(U_{10})=34.575\%$) followed by $U_{19}$ ($b^{19}(U_{19})=16.129\%)$, $U_{12}$ ($b^{19}(U_{12})=15.236\%$), and $U_{3}$ ($b^{19}(U_{3})=12.983\%$). All other universities have a frequency lower than $9\%$ of being the last.

Analogously, to pairwise compare the considered universities, we computed the pairwise winning indices shown in Table 6. Considering the three universities being most frequently in the first position, that are $U_{16}$, $U_{5}$ and $U_{11}$, one can observe that $U_{16}$ is preferred to $U_{5}$ and $U_{11}$ with frequencies of $82.043\%$ and $87.263\%$, respectively. Moreover, it is preferred to all other universities with a frequency at least equal to the 88.417%. This enforces the conviction that $U_{16}$ is the best among the universities at hand.

On the basis of the rank acceptability indices, as done also in [1], for each university we compute the best and the worst possible positions (Table 8(a)) and the most frequent ones (Table 8(b)). In this way, we have a clearer view of the goodness of the considered universities. Almost all universities have as best and worst reached positions the first and the last one. Anyway, the data in Table 8(b) give an idea on the stability of the results of each university. Indeed, $U_{16}$, $U_{5}$ and $U_{11}$ have their highest rank acceptability indices values in correspondence of the first three ranking positions. Looking at the opposite side of the ranking, $U_{10}$ and $U_{19}$ are confirmed as the worst universities since they have the highest values for their rank acceptability indices in correspondence of the last three positions. To have a look at similar results on the whole set of universities, the interested reader is deferred to the supplementary material downloadable clicking on the following link: supplementary results.

In some real world applications, the DM could wish a complete ranking of the alternatives under analysis. To summarize the results of the SMAA with a specific attention to the rank acceptability indices, [24] provided several aggregation procedures aiming to obtain a unique final ranking. In this case, we shall apply one of them in which each university $U$ will be associated to the following weighted mean of its rank acceptability indices:

[TABLE]

Looking at the results in Table 8, once again we have the proof that $U_{16}$ can be reasonably considered the best among the universities at hand, followed by $U_{11}$ and $U_{14}$, while $U_{10}$ and $U_{3}$ can be fairly considered the worst ones. A careful look at the rank acceptability indices in Table 5 and to the most frequent positions in Table 8(b) would justify the second and third positions attained by $U_{11}$ and $U_{14}$ in place of $U_{5}$. Indeed, even if $U_{5}$ reaches the first position more frequently than the other two universities, $U_{11}$ and $U_{14}$ present high frequencies for the rank acceptability indices corresponding to the second and third positions, while $U_{5}$ has a frequency lower than 10% for the same positions. Therefore, since the expected ranking takes into account all the frequencies with which an alternative reaches a certain position and not only the first one, $U_{11}$ and $U_{14}$ can be correctly considered better than $U_{5}$.

7 Conclusions

Parsimony of models in multiple criteria decision analysis has to be tempered by the possibility to permit a rich interaction with the decision maker (DM) and the consideration of robustness concerns. In this perspective we presented a decision model that has the following interesting properties:

interaction between criteria is taken into consideration,

• 2.

importance and interaction of criteria can change from one level to the other of the evaluations taken by considered alternatives on criteria at hand,

• 3.

the parameters of the model are obtained by starting from some preference information supplied by the decision makers in terms that are relatively easily understandable by them such as:

• (a)

pairwise comparisons of alternatives,

• (b)

comparisons of criteria in terms of their importance,

• (c)

nature of interaction between criteria (synergy or redundancy) and comparisons related to intensity of the interaction between pairs of criteria;

• 4.

consideration of the plurality of instances of the decision model compatible with the preferences provided by the DM so that, not only necessary preferences (valid for all compatible models) are distinguished from possible preferences (valid for at least one compatible model), but even a probabilistic evaluation of a preference of an alternative over another or the ranking position of the alternatives is given to the DM.

All this is obtained by coupling:

the level dependent Choquet integral, which permits to take into account importance and interaction of criteria that can change from one level to the other of the evaluations of criteria,

• 2.

the robust ordinal regression and the stochastic multicriteria acceptability analysis, which permit to take into account the plurality of instances of the preference model compatible with the preference information supplied by the decision makers.

We considered two possible representations of the changes of importance and interaction between criteria from one level to another of the evaluations:

the model of interval level dependent capacity, in which the range of considered evaluations is split in a certain number of subintervals and the importance and interaction of criteria changes from one subinterval to the others, but remain the same in each subinterval,

• 2.

the model of the piecewise linear level dependent capacities, in which the importance and the interaction of criteria change with continuity in the whole domain without the jumps that are present in the previous model.

Both models permit to apply Robust Ordinal Regression and Stochastic Multicriteria Acceptability Analsysis by solving LP problems and using the Hit and Run algorithm, which is a well known algorithm for sampling uniformly points in convex regions of the preference parameters space.

We believe that the good properties of the methodology we have presented can be very useful in many real life complex decision problems in the main domains of interest such as sustainable development [1], wellbeing and happiness economics [19, 20], ranking of universities [11], country competitiveness [10], and so on.

Acknowledgments

The second and the third authors wish to acknowledge funding by the FIR of the University of Catania BCAEA3 “New developments in Multiple Criteria Decision Aiding (MCDA) and their application to territorial competitiveness”