An Efficient and Truthful Pricing Mechanism for Team Formation in Crowdsourcing Markets

TL;DR

This paper introduces TruTeam, an efficient and truthful pricing mechanism for forming teams in crowdsourcing markets, ensuring incentive compatibility and computational efficiency.

Contribution

The paper proposes four incentive mechanisms for team formation, highlighting TruTeam as the most effective due to its efficiency and truthfulness.

Findings

TruTeam is computationally efficient.

TruTeam ensures truthful bidding.

Simulations confirm TruTeam's effectiveness.

Abstract

In a crowdsourcing market, a requester is looking to form a team of workers to perform a complex task that requires a variety of skills. Candidate workers advertise their certified skills and bid prices for their participation. We design four incentive mechanisms for selecting workers to form a valid team (that can complete the task) and determining each individual worker's payment. We examine profitability, individual rationality, computational efficiency, and truthfulness for each of the four mechanisms. Our analysis shows that TruTeam, one of the four mechanisms, is superior to the others, particularly due to its computational efficiency and truthfulness. Our extensive simulations confirm the analysis and demonstrate that TruTeam is an efficient and truthful pricing mechanism for team formation in crowdsourcing markets.

Tables

Table 1. TABLE I: Parameters of our two datasets

	$n$: no. of workers	$l$: no. of skills	($u,\sigma$)	$[C_1,C_2]$
Small	$n=10\sim 25$, fixing $l=5$	$l=1\sim 10$, fixing $n=20$	($l/3,0.4$)	$[1,v/5]$
Large	$n=10\sim 3000$, fixing $l=50$	$l=1\sim 100$, fixing $n=1000$	($l/5,0.4$)	$[1,v]$

Equations

$$U_R=\{\begin{array}{ll}v-{\sum }_{w_i\in S}{p}_i& \text{if task is completed}\\ 0& \text{otherwise}\end{array}$$

$$u_i=\{\begin{array}{ll}{p}_i-c_i& \text{worker wi is selected}\\ 0& \text{otherwise}\end{array}$$

$$s_T[j]=\underset{w_i\in T}{\bigvee }{s}_{w_i}[j],j=1,...,l$$

$$s_T[j]\ge s_{\ast }[j],j=1,...,l$$

$$S=\arg \underset{T}{\min }\sum _{w_i\in T}{p}_i$$

$$s.t.s_T[j]\ge s_{\ast }[j],j=1,...,l$$

$${\Delta }_i(S)=s_{S\cup \{w_i\}}\cdot s_{\ast }-s_S\cdot s_{\ast }$$

$$S=\arg \underset{T}{\min }\sum _{w_i\in T}{c}_i$$

$$s.t.s_S[j]\ge s_{\ast }[j],j=1,...,l$$

$$p_i=(\underset{T}{\min }\sum _{w_j\in T\wedge w_i\notin T}{c}_j)-\sum _{w_j\in S\wedge j\ne i}{c}_j$$

$$s.t.s_T[j]\ge s_{\ast }[j],j=1,...,l$$

$$\begin{array}{rl}{u}_i^{\prime }& =p_i^{\prime }-c_i\\ & =(\underset{T}{\min }\sum _{w_j\in T\wedge w_i\notin T}{c}_j)-\sum _{w_j\in S^{\prime }\wedge j\ne i}{c}_j-c_i\\ & =(\underset{T}{\min }\sum _{w_j\in T\wedge w_i\notin T}{c}_j)-\sum _{w_j\in S^{\prime }}{c}_j\\ & \le (\underset{T}{\min }\sum _{w_j\in T\wedge w_i\notin T}{c}_j)-\sum _{w_j\in S}{c}_j\\ & =u_i\end{array}$$

$$b_i\le \frac{b_{j_1}}{{\Delta }_{j_1}(T)}\times {\Delta }_i(T)$$

$$p_i^{\prime }=\frac{b_{j_1}}{{\Delta }_{j_1}(T)}\times {\Delta }_i(T)$$

$$p_i=\underset{j\in \{j_1,j_2,...,j_k\}}{\max }\{\frac{b_j}{{\Delta }_j(T)}\times {\Delta }_i(T)\}$$

$$\begin{array}{rl}{b}_i& \le \frac{b_{j_1}}{{\Delta }_{j_1}(S)}\times {\Delta }_i(S)\\ & \le ma{x}_{j\in \{j_1,j_2,...,j_k\}}\{\frac{b_j}{{\Delta }_j(T)}\times {\Delta }_i(T)\}\\ & =p_i\end{array}$$

Full text

I Introduction

Future crowdsourcing platforms need to support collaboration  [1]. In a collaborative crowdsourcing market, a requester is looking to form a team of workers who can perform a task that requires a set of skills. Interested candidate workers advertise their certified skills and bid prices for their participation. This differs from usual team formation problems (e.g. [2, 3, 4]) in that it not only considers the skills required by the task and possessed by the workers, but also involves economic incentives and several criteria that guarantee profitability for requesters and workers, social welfare, and truthfulness [5]. A large body of research on crowdsourcing markets have confirmed the intuition that financial incentives increase workers’ interest [6] and effort level [7] as well as the attractiveness to experienced workers [8]. However, to the best of our knowledge, existing pricing models (e.g. [9, 10, 11]) for crowdsourcing platforms only consider individual workers without taking into account teamwork which is crucial in collaborative environments [1].

In this paper, we design task allocation and pricing mechanisms for selecting a team of workers in crowdsourcing markets. We start by presenting two baseline mechanisms which pay the selected workers the same as their bids. The first baseline mechanism looks for the lowest-cost team by enumerating all possible teams in a brute force manner. This mechanism is profitable for both the requester and selected workers but is not computationally efficient. It also does not ensure truthfulness or incentive-compatibility, which means that workers may not bid their true costs but can cheat to gain higher payments. We refer to this baseline mechanism as OPT as it minimizes total payment if all workers bid truthfully. We show that OPT is profitable, individually rational but not efficient nor truthful.

The second baseline mechanism follows a greedy approach to form a team of workers with low bids and high total expertise. We refer to this mechanism as GREEDY and show that it is efficient, profitable, individually rational but not truthful.

Both baseline mechanisms do not prevent workers from overbidding, which necessitates the need of designing mechanisms in which workers will bid their true costs.

We first adapt the celebrated Vickrey-Clarke-Groves auction to our problem, and refer to this mechanism as VCG. We show that it is profitable, individually rational, truthful but not efficient.

Finally, we design a mechanism that combines the greedy selection rule and a special payment scheme. A selected worker receives a payment that equals to the highest bid she could have placed and been selected. We refer to this mechanism as TruTeam. It possesses all the above desirable properties, i.e., efficiency, profitability, individual rationality, and truthfulness.

Using synthetic scenarios, we evaluate the properties and performance of these four mechanisms. The results show that TruTeam is an efficient, truthful task allocation and pricing mechanism for team formation in crowdsourcing markets. In summary, this paper makes the following contributions:

• •

To the best of our knowledge, this is the first study on team formation in collaborative crowdsourcing markets.

• •

We formulate the problem of team formation in crowdsourcing as an task allocation and pricing mechanism design problem.

• •

We design two baseline and two refined mechanisms and prove profitability, individual rationality, prove or disprove truthfulness for each of the four mechanisms. In addition, we also evaluate their computational complexity.

• •

We show via both analysis and extensive simulations that TruTeam is an efficient, profitable, individually rational and truthful mechanism for team formation in crowdsourcing markets.

II Related Work

II-A Pricing Mechanisms

Various models and techniques have been proposed for pricing on crowdsourcing platforms. Budget-feasible mechanisms [12] maximize a requester’s profit under a budget constraint while satisfying other properties such as truthful bidding. In [9], the authors designed a framework for task allocation and pricing in an online environment. The framework aims to maximize the number of tasks performed under a given budget or to minimize payments for a given number of tasks. The authors of [10] designed a no-regret posted price mechanism which bridges the gap between procurement auctions and multi-armed bandits. That mechanism satisfies budget feasibility, achieves near-optimal utility for the requester, and also guarantees that workers bid their true costs. In [11], workers and tasks are modeled as a bipartite graph where an edge $(m,n)$ in the graph represents worker $n$ is willing to perform task $m$. The authors designed a payment mechanism that ensures budget feasibility and one-way-truthfulness while achieving near-optimal utility. Recent work [13] considers variable rather than fixed payment in crowdsourcing, as a function of the best worker’s effort, and achieves a higher utility than the optimum achieved by fixed payment. This idea was then extended to heterogeneous crowdsourcing environments [14] to tackle non-uniform knowledge possessed by different workers.

II-B Task Allocation and Team Formation

The task allocation problem is related to the job scheduling problem which aims at minimizing the load of the machines that have maximal work load. An extended version of the job scheduling problem, in which each job needs to be performed on a set of machines, was proved to be NP-hard [15].

The authors of [16] studied a new variant of the task allocation problem in which the workers are connected in a social network. The workers are assumed to only have local knowledge about resources and hence each task can only be assigned to its neighboring workers. The authors proved this problem to be NP-hard and proposed a max-flow network model to solve it.

Team formation in a non-crowdsourcing environment (e.g. social networks) was studied by [17, 4], assuming workers need to communicate when performing a task and communication is costly. The selected team has the skills to complete a task and has minimum communication cost. A different metric, the workload of workers, was studied by [2] when selecting a team to complete a given task. The authors of [3] considered both workload and communication costs when selecting a team to perform a task.

In this paper, we consider pricing mechanisms for the team formation problem, which bridges the gap between budget-feasible mechanisms and the traditional team formation problem. We do not consider workload or communication costs but rather focus on truthfulness which is more important in crowdsourcing environments. In addition, our problem is more general than prior work on budget-feasible mechanisms where a task is always assigned to a single worker.

III Model

III-A Requester and Workers

In our model, a single requester posts her task to a crowdsourcing platform. The task has a value $v$ which is the requester’s revenue if the task is completed. There is a set of $n$ available workers $W=\{w_{1},...,w_{n}\}$, and the task needs a subset of workers, $S\subseteq W$, to collaborate. When a worker signs up to participate in the task, she should report to the requester what skills she has and how much she expects to be paid. Then the requester selects a set of workers and decides on the payment for each of them.

Cost and bid of a worker. We assume that each worker’s cost of doing the task is private information. Each worker $w_{i}$ has a non-negative cost $c_{i}\in\mathcal{R}_{\geq 0}$ to perform the task, and bids $b_{i}\in\mathcal{R}_{\geq 0}$ when she signs up to do the task.

We assume that a worker cannot lie about the skills that she has. Some existing crowdsourcing platforms such as MTurk [18] provided qualification test to ensure the validity of workers’ skills. A platform can also use work history information to verify one’s skill.

Utility of the requester. The requester’s utility $U_{R}$ is the revenue obtained from the completed task, subtracting the payment to the selected workers.

[TABLE]

where $S$ is the set of selected workers and $p_{i}$ is the payment to worker $w_{i}$.

Utility of a worker. Worker $w_{i}$’s utility $u_{i}$ is the payment she receives subtracting her cost of performing the task.

[TABLE]

Both the requester and the workers aim to maximize their respective utilities.

III-B Skill Profiles

The skill profile of the given task is an $l$-dimension vector $s_{*}=(s_{*}[1],...,s_{*}[l])$, where $s_{*}[i]=\{0,1\}$ represents that the $i_{th}$ skill is required $(1)$ or not $(0)$. We assume that a maximum of $l$ skills are required for any task. The skill profile of a worker $w_{i}$ is also an $l$-dimension vector $s_{w_{i}}=(s_{w_{i}}[1],...,s_{w_{i}}[l])$, similarly defined but representing what skills $w_{i}$ possesses.

The skill profile of a team $s_{T}=(s_{T}[1],...,s_{T}[l])$ is defined by a logical OR of the skill profiles of all the individual workers in the team $T$:

[TABLE]

The team that can complete the task is the team that has all the required skills of the task, i.e.,

[TABLE]

III-C Desirable Properties

Computational Efficiency. The task allocation and pricing mechanism can be executed in polynomial time.

Individual Rationality. No worker is worse off if she is selected to do the task. In other words, each selected worker receives a payment no less than her true cost.

Profitability. The utility of the requester is non-negative.

Truthfulness (Incentive Compatibility). Bidding her true cost is each worker’s dominant strategy. Formally, if $u_{i}$ and $u_{i}^{\prime}$ are the utilities of worker $w_{i}$ when bidding truthfully and untruthfully, respectively, then a truthful mechanism guarantees that $u_{i}\geq u_{i}^{\prime}$ regardless of what other workers bid.

Theorem 1**.**

An auction mechanism is truthful if and only if [12]:

• •

The allocation rule is monotone: If worker $w_{i}$ wins the auction by bidding $b_{i}$, she also wins by bidding $b_{i}^{\prime}\leq b_{i}$.

• •

Each winner is paid the threshold price: Worker $w_{i}$ will not win the auction if she bids higher than this price.

III-D Design Objectives

We want to design a task allocation and pricing mechanism for the requester to select a team to complete a task she posts, with the objective of minimizing the total payment to the selected workers, i.e.,

[TABLE]

[TABLE]

In addition, the mechanism should satisfy computational efficiency, individual rationality, profitability and truthfulness.

IV Mechanisms

IV-A Optimal Mechanism

This mechanism selects the team with the lowest total bid that is able to complete the task, by taking a brute-force approach to attempt all the possible $2^{n}-1$ teams (excluding the empty set) of the $n$ workers. We refer to this mechanism as OPT.

** Lemma 1****.**

OPT is individually rational, profitable, but not computationally efficient or truthful. Time complexity of OPT is $O(2^{n})$.

The proof is deferred to [19] due to space constraint, as is also the case for Lemma 2 and Theorem 2.

IV-B Greedy Mechanism

This heuristic mechanism selects the worker with the minimum cost per marginal skill contribution until a team that can complete the task is formed or all the workers have been considered. It pays each selected worker her bid.

In the above, $w_{i}$’s marginal skill contribution, $\Delta_{i}(S)$, is defined with respect to an existing worker set $S$ as the number of uncovered skills that $w_{i}$ can cover if selected into $S$:

[TABLE]

where $s_{S}$ is the skill profile of team $S$, and $s_{S}\cdot s_{*}$ is the inner product of vectors $s_{S}$ and $s_{*}$. In each iteration, the GREEDY mechanism always selects the worker who has the lowest cost per marginal skill contribution, $\frac{b_{i}}{\Delta_{i}(S)}$.

** Lemma 2****.**

GREEDY is computationally efficient, individually rational, profitable, but not truthful. Time complexity of GREEDY is $O(n^{2})$.

IV-C VCG-based Mechanism

We adapt the traditional Vickrey-Clarke-Groves (VCG) mechanism [5] to our problem, and design a mechanism called VCG.

Allocation rule. VCG selects the team with the lowest total cost, i.e.,

[TABLE]

[TABLE]

Payment rule. VCG pays each selected worker $w_{i}$ the difference between the optimal welfare (for the other workers) if $w_{i}$ was not participating and welfare of the other workers with respect to the selected team:

[TABLE]

[TABLE]

where $S$ is defined in (8).

** Lemma 3****.**

VCG is individually rational, profitable, truthful, but not computationally efficient. Time complexity of VCG is $O(n2^{n})$.

Proof.

Suppose $u_{i},u_{i}^{\prime}$, $p_{i},p_{i}^{\prime}$ are the corresponding utilities and payments of $w_{i}$, when she bids truthfully and untruthfully, respectively. Let $S,S^{\prime}$ denote the selected team when $w_{i}$ bids truthfully and untruthfully, respectively.

[TABLE]

which proves the truthfulness. The proof of other properties are deferred to [19] due to space constraint. ∎

IV-D Efficient and Truthful Mechanism (TruTeam)

OPT is an optimal allocation mechanism if every worker bids truthfully. GREEDY is computationally efficient but not a truthful mechanism. VCG is a truthful mechanism but is not computationally efficient.

In this section, we present the mechanism TruTeam (Mechanism 1) which satisfies all the four properties (i.e., TruTeam is computationally efficient, individually rational, profitable, and truthful).

Allocation rule. In each iteration, it selects the worker who has the smallest cost per marginal skill contribution, i.e., the lowest $\frac{b_{i}}{\Delta_{i}(S)}$. This is the same as GREEDY.

Payment rule. The intuition of the payment rule is to pay each selected worker the highest cost she can report while still being selected [11]. This is the “threshold price” stated in Theorem 1, and we will show that overbidding under TruTeam does no good to improve a worker’s utility.

Now, we explain in detail how to determine the payment to each selected worker. When computing the payment to worker $w_{i}$, let’s see how this mechanism selects a team without $w_{i}$’s participation. It selects from set $W\backslash\{{T\cup\{w_{i}\}}\}$ ($T$ is the selected worker set before $w_{i}$) the worker ($w_{j_{1}}$) who minimizes the value $\frac{b_{j}}{\Delta_{j}(S)}$. Therefore

[TABLE]

Otherwise, if $\frac{b_{i}}{\Delta_{i}(T)}>\frac{b_{j_{1}}}{\Delta_{j_{1}}(T)}$, we would have selected $w_{j_{1}}$ instead of $w_{i}$ according to the allocation rule of this mechanism. Therefore, we set the payment to worker $w_{i}$ equal to this value:

[TABLE]

However, $p_{i}^{\prime}$ may not be the highest bid that $w_{i}$ can report while still being able to be selected, because $T\cup\{w_{j_{1}}\}$ may not cover all the skills required by the task. Suppose $T\cup\{w_{j_{1}},w_{j_{2}},...,w_{j_{k}}\}$ (without $w_{i}$) is the set of workers selected according to this mechanism. We set the payment equal to the following threshold price as described in lines 6-10.

[TABLE]

Note that, $T$ is updated every time by including a new worker $w_{j_{x}}$ ($x=1,2,...,k$), i.e., $T=T\cup\{w_{j_{x}}\}$.

In order to be profitable, $w_{i}$ is selected to perform this task only if the task’s remaining value is not less than $p_{i}$ (we also update the set of selected workers as $S=S\cup\{w_{i}\}$), as shown in lines 11-13. Otherwise we skip $w_{i}$ and consider next candidate worker as described in lines 14-15.

Repeat the above process until the task can be completed by the set of workers $S$ or all the workers have been considered.

** Lemma 4****.**

TruTeam is computationally efficient, with a time complexity of $O(n^{2}l)$.

Proof.

Selecting the worker who has the minimal value $\frac{b_{i}}{\Delta_{i}(S)}$ takes $O(n)$ time. Deciding the payment for the selected worker takes $O(nl)$. Since there are $n$ workers, time complexity of this mechanism is $O(n^{2}l)$. ∎

** Lemma 5****.**

TruTeam is individually rational.

Proof.

From the above payment rule, we can see that for any selected worker $w_{i}$.

[TABLE]

We assume a worker will not bid bellow her true cost, i.e. $b_{i}\geq c_{i}$ (In fact, we will show in Lemma 7 that $b_{i}=c_{i}$ ). Therefore, $w_{i}$’s utility is $u_{i}=p_{i}-c_{i}\geq p_{i}-b_{i}\geq 0$. ∎

** Lemma 6****.**

TruTeam is profitable.

Proof.

Every time when a worker is considered, we check if the remaining value of the task can cover the payment to this worker. If not, the worker will not be selected. This process guarantees that the value of this task is more than the total payment to the whole team. ∎

** Lemma 7****.**

TruTeam is truthful.

Proof.

According to Theorem 1, we need to prove that (1) the allocation rule is monotone and (2) the payment to each selected worker $w_{i}$ is the threshold price $p_{i}$.

The monotonicity of the allocation rule is obvious, since if $w_{i}$ is selected, she will also be selected by bidding a smaller value which leads to a smaller cost per marginal skill contribution.

For the threshold price, recall $p_{i}$ from equation (13). If $b_{i}>p_{i}$, $w_{i}$ will be placed after the last selected worker, thus she will not be selected to perform the task. Therefore, $p_{i}$ is the threshold price.111Bidders will not underbid, either. The intuition is that if $w_{i}$ underbids and is selected, her payment will not cover her cost. See [19] for the details. ∎

Theorem 2**.**

TruTeam is computationally efficient, individually rational, profitable and truthful.

V Evaluation

In this section, we compare the performance of the four mechanisms, namely OPT, VCG, GREEDY and TruTeam in terms of the following metrics:

• •

Running Time: the actual CPU time on a computer.

• •

Requester’s Utility: defined in Eqn. (1).

• •

Truthfulness: We verify the truthfulness of TruTeam by evaluating workers’ utility if they overbid or underbid.

V-A Simulation Setup

We generate two different datasets to evaluate our mechanisms. A Small dataset is used to evaluate all the four mechanisms and a Large dataset is used to evaluate the two computationally efficient mechanisms (i.e., GREEDY and TruTeam). The parameters of the two datasets are listed in Table I and explained bellow.

We set the value of the task $v=500$ which is unknown to workers. Each worker’s true cost $c_{i}$ is uniformly drawn from $[C_{1},C_{2}]$. In the case that all workers are truthful, we set $b_{i}=c_{i}$. In the case of overbidding, we randomly select $k\in[1,n]$ workers and let each of them overbid a random value $r_{i}$ (i.e., $b_{i}=c_{i}+r_{i}$) where $r_{i}\in[1,v]$. We do not consider underbidding, since no rational worker will underbid under these four mechanisms.

To generate each worker’s skill profile, we first generate the number of skills she has, using the normal distribution ($u,\sigma$). Suppose $w_{i}$ has $x$ skills, then we randomly assign $x$ different skills out of all the $l$ skills to her.

All the simulations were run on a Windows PC with a 3.40GHz CPU and 8 GB memory. Each data point is averaged over 100 measurements.

V-B Results

Fig. 1 shows the comparison of the four mechanisms conducted over the Small dataset. We observe in Fig. 1 (a,b,c,d) that OPT and VCG do not scale well when the number of workers or skills becomes large. However, all the four mechanisms achieve strictly positive requester’s utility (e,f,g,h). Therefore, in this section, we focus on the computationally efficient mechanisms, GREEDY and TruTeam, and explain in detail the results pertaining to them collected on the Large dataset.

V-B1 Running Time

The results are presented in Fig. 2 (a,b,c,d). Generally, as the number of workers or the number of skills increases, running time of both mechanisms increases. Although GREEDY outperforms TruTeam, since the payment determination process of TruTeam is more complicated than that of GREEDY, TruTeam maintains a high efficiency. For example, it completes in about 0.6 second even when the number of worker reaches 3000. More importantly, TruTeam ensures truthfulness which is crucial to incentive mechanisms to counteract possible cheating behaviors in practice.

In Fig. 2(a) and 2(b), running time of TruTeam contains a small peak when the number of workers is between 100 and 300. This happens when the task is complex (requiring skills as many as $l=50$) and the number of workers is relatively small ($n<300$). Thus the team size is fairly large and TruTeam needs to check every other worker’s bid and skills when deciding the payment to each selected worker, resulting in higher running time.

V-B2 Requester’s Utility

Fig. 2 (e,f,g,h) present the requester’s utility in different settings. Generally, the requester’s utility increases as the number of worker increases (e,f), since the requester has more “cheaper” workers to select from. On the other hand, the requester’s utility drops as the number of required skills becomes larger (g,h), which is a natural result of the increase of complexity of the task.

In the case of truthful bidding, as is shown in Fig. 2(e) 2(g), the untruthful mechanism (GREEDY) yields higher utility than the truthful mechanism (TruTeam) because TruTeam pays each selected worker no less than her bid.

However, in the case of overbidding which is a more realistic setting, Fig. 2(f) 2(h) show that TruTeam outperforms GREEDY. This demonstrates that in real crowdsourcing markets where workers are strategic and speculating higher payment (e.g., by trying to overbid), TruTeam generates higher profit for the requester by ensuring truthful bidding.

V-B3 Truthfulness

Lastly, we verify the truthfulness of TruTeam by examining the utilities of two randomly chosen workers, $w_{637}$ and $w_{219}$. We set $n=1000$ and $l=50$, and their true costs are $c_{637}=4$ and $c_{219}=10$, respectively. In Fig. 3(a), we observe that $w_{637}$ is selected to perform the task if she bids her true cost $c_{637}=4$, and her utility reaches the optimal value $7$. If she overbids a value no less than 11, she is not selected and therefore her utility drops to 0. In Fig. 3(b), it is observed that $w_{219}$ is not selected to do the task if she bids her true cost $c_{219}=10$, and hence her utility is [math]. This is the optimal utility she can get because even though she can be selected to do the task, which only happens if she under bids (below 7), her payment will not be able to cover her true cost and hence she will receive a negative utility, as indicated in Fig. 3(b).

TruTeam ensures that it is every worker’s dominant strategy to bid her true cost in order to maximize her utility.

VI Conclusion

In this paper, we formulate team formation in crowdsourcing as a task allocation and pricing problem, and provide four candidate mechanisms as solutions: OPT, GREEDY, VCG, and TruTeam. We prove that although all the four mechanisms satisfy profitability and individual rationality, only VCG and TruTeam satisfy truthfulness, and only GREEDY and TruTeam are efficient. Simulation also demonstrate that TruTeam is the only one, among the four candidates, that is efficient, profitable, individually rational and truthful.

To the best of our knowledge, this is the first study on team formation in collaborative crowdsourcing markets. Future research could be in the direction of taking into account the quality of contribution [20] and trustworthiness of workers [21] when selecting and rewarding workers, or considering previous collaborations among workers and inter-dependency among multiple tasks when forming multiple teams.

Acknowledgment

This work is supported by the French Ministry of European and Foreign Affairs under the STIC-Asia program, CCIPX project.