A Real-Time Framework for Task Assignment in Hyperlocal Spatial Crowdsourcing

TL;DR

This paper presents a real-time framework for task assignment in hyperlocal spatial crowdsourcing, addressing budget constraints and dynamic worker and task arrivals to optimize task coverage.

Contribution

It introduces a taxonomy of problem variants, analyzes their complexity, and proposes online heuristics leveraging spatial-temporal knowledge for effective task assignment.

Findings

Heuristics outperform baseline methods in experiments.

Proposed solutions efficiently handle dynamic arrivals.

Framework adapts to various budget and utility scenarios.

Abstract

Spatial Crowdsourcing (SC) is a novel platform that engages individuals in the act of collecting various types of spatial data. This method of data collection can significantly reduce cost and turnover time, and is particularly useful in urban environmental sensing, where traditional means fail to provide fine-grained field data. In this study, we introduce hyperlocal spatial crowdsourcing, where all workers who are located within the spatiotemporal vicinity of a task are eligible to perform the task, e.g., reporting the precipitation level at their area and time. In this setting, there is often a budget constraint, either for every time period or for the entire campaign, on the number of workers to activate to perform tasks. The challenge is thus to maximize the number of assigned tasks under the budget constraint, despite the dynamic arrivals of workers and tasks. We introduce a taxonomy of several problem variants, such as budget-per-time-period vs. budget-per-campaign and binary-utility vs. distance-based-utility. We study the hardness of the task assignment problem in the offline setting and propose online heuristics which exploits the spatial and temporal knowledge acquired over time. Our experiments are conducted with spatial crowdsourcing workloads generated by the SCAWG tool and extensive results show the effectiveness and efficiency of our proposed solutions.

Tables

Table 1. Table 1: The coverage instance set of the example in Figure  2 .

Time	Coverage Instance Sets
1	$\{(w_1^1,$$t_1^1,t_1^2,t_1^3$$>$$),(w_1^2,$$t_1^1,t_1^4,t_1^5,t_1^6$$>$$\left)\right\}$
2	$\{(w_2^1,$$t_1^5,t_1^6$$>$$\left)\right\}$

Table 2. Table 2: Summaries of real-world datasets.

Name	#Tasks	#Workers	MTD 121212MTD: Mean Travel Distance [To et al. (2014)]	$\left|s_i\right|$
Foursquare	89,968	45,138 (90/$k{m}^2$)	16.6km	1 hour
Gowalla	151,075	6,160 (35/$k{m}^2$)	3.6km	1 day

Equations

$$s_i<t_i^k.(s+\delta )$$

$$||w_i^j.l-t_i^k.l|{|}_2\le t_i^k.r$$

$$priority(w_i^j)=\sum _{t_i^k\in C(w_i^j)\cap R}\frac{1}{t_i^k.(s+\delta )-i}$$

$$RE(t)=-\sum _{w\in W_t}{P}_t(w)\times log{P}_t(w)$$

$$priority(w_i^j)=\sum _{t_i^k\in C(w_i^j)\cap R}\frac{1}{1+RE(t_i^k)}$$

$${\delta }_K=\sum _{t=1}^i(K^{base}[t])-K_{used}$$

$${\delta }_{\lambda }={\lambda }_l-\overline{{\lambda }_{l-1}}$$

$$coun{t}_i(w_{id})=\sum _{k=1}^i\sum _{j=1}^{|W_k|}d(w_k^j)[w_k^j.id=id]$$

$$Maximize\bigcup _{w_i^j\in W_i}C(w_i^j)$$

$$Minimize\underset{w_i^j\in W_i}{\max }(coun{t}_i(w_i^j))$$

$$s.t.\sum _{j=1}^{|W_i|}d(w_i^j)\le K_i$$

$$\alpha |\bigcup C(w_i^j)|/|T_i|-(1-\alpha )\max (count(w_i^j))/Q$$

$$\sum _i\sum _{j=1}^{|W_i|}d(w_i^j)\le K$$

$${\lambda }_l=\alpha .priority(w_i^j)/|T_i|-(1-\alpha )count(w_i^j)/Q$$

$$priority(w_i^j)=\sum _{t_i^k\in C(w_i^j)\cap R}f(dist(t_i^k,w_i^j))$$

$$priority(w_i^j)=\sum _{t_i^k\in C(w_i^j)\cap R}\frac{f(dist(t_i^k,w_i^j))}{t_i^k.(\delta +s)-i}$$

$$priority(w_i^j)=\sum _{t_i^k\in C(w_i^j)\cap R}\frac{f(dist(t_i^k,w_i^j))}{1+RE(t_i^k)}$$

Full text

{CCSXML}

¡ccs2012¿ ¡concept¿ ¡concept_id¿10002951.10003260.10003282.10003296¡/concept_id¿ ¡concept_desc¿Information systems Crowdsourcing¡/concept_desc¿ ¡concept_significance¿500¡/concept_significance¿ ¡/concept¿ ¡concept¿ ¡concept_id¿10003120.10003138¡/concept_id¿ ¡concept_desc¿Human-centered computing Ubiquitous and mobile computing¡/concept_desc¿ ¡concept_significance¿500¡/concept_significance¿ ¡/concept¿ ¡concept¿ ¡concept_id¿10002951.10003227.10003236.10003237¡/concept_id¿ ¡concept_desc¿Information systems Geographic information systems¡/concept_desc¿ ¡concept_significance¿500¡/concept_significance¿ ¡/concept¿ ¡/ccs2012¿

\ccsdesc

[500]Information systems Crowdsourcing \ccsdesc[500]Human-centered computing Ubiquitous and mobile computing \ccsdesc[500]Information systems Geographic information systems

\acmformat

Luan Tran, Hien To, Liyue Fan, Cyrus Shahabi, 2017. A Real-Time Framework for Task Assignment in Hyperlocal Spatial Crowdsourcing.

1 Introduction

With the ubiquity of smart phones and the improvements of wireless network bandwidth, every person with a mobile phone can now act as a multimodal sensor collecting and sharing various types of high-fidelity spatiotemporal data instantaneously. In particular, crowdsourcing for weather information has become popular. With a few recent apps, such as mPING222http://mping.nssl.noaa.gov/ and WeatherSignal333http://weathersignal.com, individual users can report weather conditions, air pollutions, noise levels, etc. In fact, the authors in [Dorminey (2014)] regards crowdsourcing as “the future of weather forecasting”.

Through our collaboration with the Center for Hydrometeorology and Remote Sensing (CHRS)444http://chrs.web.uci.edu/ at the University of California, Irvine, we have developed a mobile app, iRain555https://itunes.apple.com/us/app/irain-uci/id982858283 [iRa (2016)], to perform spatial crowdsourcing for precipitation information. Unlike other weather crowdsourcing apps, iRain allows CHRS researchers to request rainfall information at specific locations and times where their global satellite precipitation estimation technologies666http://hydis.eng.uci.edu/gwadi/ fail to provide real-time, fine-grained data. Individual iRain users around those locations can respond to those requests by reporting rainfall observations, e.g., heavy/medium/light/none.

In general, spatial crowdsourcing (SC) [Kazemi and Shahabi (2012)] offers an effective data collection platform where data requesters can create spatial tasks dynamically and workers are assigned to tasks based on their locations. Figure 1 depicts the architecture of iRain. A requester issues a set of rainfall observation tasks to the SC-server (Step 1) where each task corresponds to a specific geographical extent, e.g., a circle. The workers continuously update their locations to the SC-server when they become available for performing tasks (Step 0). Subsequently, the SC-server crowdsources the tasks among the workers in the task regions and sends the collected data back to the requester (Steps 2, 3).

One major difference from existing SC paradigms [Kazemi and Shahabi (2012), He et al. (2014), To et al. (2015), Xiao et al. (2015), Guo et al. (2016)] is that workers in our paradigm do not need to travel to the exact task locations, e.g., to the centers of the circular regions, and are eligible to perform tasks as long as they are in close spatiotemporal vicinity of the tasks, e.g., enclosed in the circular regions777Tasks that require workers to physically travel to task locations, e.g., taking a picture of an event, are not considered in our problem setting.. We denote this new paradigm as Hyperlocal Spatial Crowdsourcing. The reason is two-fold. Without requiring the workers travel physically, our paradigm lowers the threshold for worker participation and will potentially yield faster response. Furthermore, the requested data, e.g., rainfall or temperature, exhibits spatiotemporal continuity in measurement. Therefore, observations obtained at nearby locations, e.g., within a certain distance to the task location, and close to the requested time, are sufficient to fulfil the task. For example, workers $B$ and $C$ in Figure 1 are both eligible to report precipitation level at University of Southern California (USC), and worker $A$ who becomes available 5 minutes later is also qualified. The acceptable ranges of space and time can be specified by data requesters, from which the SC-server can find the set of eligible workers for each task.

The SC-server operates to maximize fulfilled tasks for revenue. Therefore it cannot assign tasks to an unlimited number of workers due to budget considerations. In Hyperlocal SC, the budget represents the payment to each selected worker upon task completion, or the communication cost for sending/receiving task notifications between the SC-server and each selected worker. Furthermore, it is not necessary to select many workers for overlapping tasks. For example in Figure 1, the observation of worker $A$ can be used for precipitation tasks at both USC and Los Angeles downtown (shown in two circles).

The goal of our study is to maximize the number of assigned tasks on the SC-server where only a given number of workers can be selected over a time period or during the entire campaign, i.e., under “budget” constraints. When tasks and workers are known a priori, we can reduce the task assignment problem to the classic Maximum Coverage Problem and its variants. However, the main challenge with SC comes from the dynamism of the arriving tasks and workers, which renders an optimal solution infeasible in the online scenario. In Figure 1, the SC-server is likely to activate worker $D$ and either worker $B$ or $C$ for the two tasks, respectively, without knowing that a more favorable worker $A$ is qualified for both tasks and will arrive in the near future. Previous heuristics in the literature [Kazemi and Shahabi (2012), To et al. (2015), Deng et al. (2016), Guo et al. (2016)] do not consider the vicinity of tasks in space and time or the budget, thus cannot be applied to Hyperlocal SC.

The contributions of this paper are as follows888This paper is an extension of a short paper appeared in [To et al. (2016b)]. 1) We provide a formal definition of Hyperlocal Spatial Crowdsourcing, where the goal is to maximize task coverage under budget constraints. We introduce a taxonomy to classify several problem variants, e.g., given a budget constraint for each time period (${f\textit{MTC}}$) vs. for the entire campaign (${d\textit{MTC}}$). We show that both ${f\textit{MTC}}$ and ${d\textit{MTC}}$ variants are NP-hard in the offline setting. 2) In the online setting, we propose several heuristics for real-time task assignment. When a budget constraint is given for each time period (${f\textit{MTC}}$), local heuristics i.e., Basic, Temporal, and Spatial, are developed to select workers within each time period. When a budget is given for the entire campaign (${d\textit{MTC}}$), we devise an adaptive strategy based on the contextual bandit to dynamically allocate the total budget to a number of time periods. 3) When the utility of an assigned task is considered, we introduce two distance-based utility models to measure the assignment quality based on worker-task distance, which can be integrated with any previously developed heuristics. To avoid overloading workers, we introduce a multi-objective variant in order to minimize the repetitive activation of the same worker. Online solutions based on genetic algorithm and adaptive budget allocation are developed for ${f\textit{MTC}}$ and ${d\textit{MTC}}$ scenarios, respectively. 4) We conduct extensive experiments with workbench datasets generated from real-world location check-ins. The empirical results confirm that our heuristics are efficient and effective in assigning hyperlocal tasks in a real-time manner.

The remainder of this paper is organized as follows. Section 2 reviews the related work. Section 3 provides notations and a taxonomy for Hyperlocal SC problem. We prove in Section 4 that offline task assignment in Hyperlocal SC with budget constraints is NP-hard. In Section 5, we study two problem variants in the online setting. Section 6 discusses the multi-objective optimization variant to mitigate worker overloading and Section 7 describes the integration of distance-based task utility models. We report our experimental results in Section 8, provide discussion in Section 9, and conclude the paper in Section 10.

2 Related work

Spatial Crowdsourcing (SC) can be deemed as one of the main enablers of urban computing’s applications such as monitoring traffic information and air pollution [Zheng et al. (2014), Ji et al. (2016)]. Only recently SC has gained popularity in both research community and industry, e.g., TaskRabbit, Gigwalk. A recent study [To et al. (2015)] distinguishes SC from related fields, including generic crowdsourcing, participatory sensing, volunteered geographic information, and online matching. Research efforts in SC have focused on different aspects, such as task assignment (e.g., [Kazemi and Shahabi (2012), Tong et al. (2016), Liu et al. (2016b), Cheng et al. (2016), Hu et al. (2016)]), task scheduling (e.g., [Deng et al. (2016), Li et al. (2015), Sales Fonteles et al. (2016)]), quality control and trust (e.g.,  [Kazemi et al. (2013), Cheng et al. (2015)]), privacy (e.g., [To et al. (2014), To et al. (2017), Wang et al. (2017)]), incentive mechanism (e.g., [Gao et al. (2015), Kandappu et al. (2016), Zhang et al. (2017)]). The authors in [Kazemi and Shahabi (2012)] proposed task assignment problem whose goal is to maximize the number of assigned tasks. Requesters may want to crowdsource a spatial complex task that requires multiple workers at different locations to collectively perform several spatial sub-tasks [Dang et al. (2013), Zhang et al. (2016), Gao et al. (2017)]. In [To et al. (2014)], the authors introduce the problem of protecting worker location privacy in SC. A framework is proposed to ensure differentially-private protection guarantees without significantly affecting the effectiveness and efficiency of the SC system. Another study proposes a differentially private incentive mechanism in mobile crowd sensing system [Jin et al. (2016)]. In such systems where workers bid tasks, worker’s bid may reveal her interests, knowledge base. Thus, the proposed method preserves the privacy of each worker’ bid against the other honest-but-curious workers. However, unlike our study, this work does not focus on challenges that are unique to spatial crowdsourcing such as spatial task allocation. Meanwhile, the authors in [Pournajaf et al. (2014)] propose a two-phase framework whose objective is to match a set of spatial tasks to a set of workers given the workers’ cloaking regions such that task assignment is maximized while satisfying travel budget constraint of each worker. The trust issues in SC have been studied in [Kazemi et al. (2013)], where one solution is having tasks performed redundantly by multiple workers. Recently in [Cheng et al. (2016)], the reliability of task assignment is measured in term of both the confidence of task completion and the diversity quality of the tasks. However, the trust and reliability issues of workers are beyond the scope of our work; if there are multiple reports for one task, the SC-server will simply send all available reports to the task requester. It is worth noting that we assume the workers would respond to their assigned tasks, which is a common assumption in the server-assigned mode of spatial crowdsourcing [Kazemi and Shahabi (2012)]. There have been studies that relax this assumption by associating to each worker a probability to perform an assigned task (e.g., [Cheng et al. (2016), To et al. (2014)]). However, this consideration is not the focus of our paper. In addition, the time required for the workers responding to an assigned task is negligible (e.g., a few seconds with the iRain application) when compared to the deadline of each task (e.g., one day).

Online Spatial Task Assignment: There have been extensive studies regarding task assignment in generic crowdsourcing (e.g., [Venanzi et al. (2013), Tran-Thanh et al. (2013)]). However, unlike our study that focuses on the spatiotemporal aspects of the task assignment, these studies focus on task assignment in general crowdsourcing rather than spatial crowdsourcing. Recent studies that are closely related to ours include [ul Hassan and Curry (2014)] and [Tong et al. (2016)]. Both of them study the online spatial task assignment problem; however, they differ from our work in terms of the problem setting and objectives. First, in our problem the report of a worker can be used for multiple tasks as long as their geographical extents cover the worker’s location. Thus, our focus is worker selection rather than matching of workers to tasks as in [ul Hassan and Curry (2014), Tong et al. (2016)]. Second, the objectives in these studies are respectively to maximize the number of assigned tasks and to maximize the total utility score of the worker-task matches while our framework considers both kinds of utility, each is jointly combined with other real-world considerations, i.e., leveraging historical workload and minimizing worker overloading. In addition, our study maximizes the utility of assignment under budget constraints while others focus solely on maximizing the utility.

Budgeted Spatial Task Assignment: There have been recent studies on matching workers with tasks under budget constraints. In [Tran-Thanh et al. (2013)], the authors propose CrowdBudget — an agent-based budget allocation algorithm that divides a given budget among different tasks in order to achieve low estimation error (of the estimated answers for a set of tasks). This study does not consider the challenges of spatial task assignment, where workers and tasks can come and go at any time and we are not aware of their locations until their arrival time. The study in [Miao et al. (2016)] also differs from ours. In our problem, the workers do not need to travel and report sensed value at their current locations. In contrast, in [Miao et al. (2016)], the workers do need to travel to the task locations, which may take a long time in rush hour. Consequently, the workers may reject their assigned tasks. It is worth noting that these studies focus on worker/task matching problem; however, our aim is to select the best workers to maximize task coverage. Similar to [Zhang et al. (2015), Tran-Thanh et al. (2013)], the budget in [Miao et al. (2016)] refers to payment to workers, while in our study, we consider budget as the number of workers to select and focus on allocating a total budget across multiple time periods.

Worker Selection: Several works studied the problem of selecting workers with budget constraints [Song et al. (2014), Zhang et al. (2014)]. However, those studies focus on offline participant selection problem while our focus is to propose online solutions. Furthermore, the problem settings in those studies differ from ours in several aspects. Sensing tasks in [Song et al. (2014)] are represented by non-overlapping regions while tasks in our study can overlap spatially thus more challenging for optimization. The authors in [Zhang et al. (2014)] studied the problem of selecting a minimum number of workers to minimize the overall incentive payment while satisfying a probabilistic coverage requirement; however, in our problem, the number of workers to be selected is constrained by a predefined budget. Our work is also closely related to the problem of matching workers with tasks [He et al. (2014), Xiao et al. (2015)]. Particularly, in [He et al. (2014)], the authors studied the problem of task allocation that maximizes the reward of the SC platform given a time constraint for each worker. Recently in [Xiao et al. (2015)], a task assignment problem that minimizes the average makespan of all assigned tasks was proposed. Unlike these studies, SC workers in our setting need not travel to task locations. Furthermore, our aim is different from the aforementioned studies, which is to maximize task coverage.

3 Preliminaries

We first introduce concepts and notations used in this paper. A task is a query of certain hyperlocal information, e.g., precipitation level at a particular location and time. For simplicity, we assume that the result of a task is in the form of a numerical value, e.g., 0=rain,1=snow,2=none999Remote sensing techniques based on satellite images cannot differentiate between rain and snow.. Specifically, every task comes with a pre-defined region where any enclosed worker can report data for that task. In this paper, we define each task region as a circular space centered at the task location; however, task region can be extended to other shapes such as a polygon or to represent geography such as district, city, county, etc. Moreover, each task also specifies a valid time interval during which users can provide data.

Definition 3.1** (Task).**

A task $t$ of form <$${l,r,s,\delta}$$> is a query at location $l$, which can be answered by workers within a circular space centered at $l$ with radius $r$. The parameter $\delta$ indicates the duration of the query: it is requested at time $s$ and can be answered until time $s+\delta$.

We refer to $s+\delta$ as the “deadline” of task $t$. A task expires if it has not been answered before its deadline. Figure 2(a) shows the regions of six tasks, $t_{1}^{1},t_{1}^{2},...,t_{1}^{6}$. All tasks expire at time period 2 (i.e., they can be deferred to time period 2), represented by the dashed circles in Figure 2(b).

A worker can accept task assignments when he is online.

Definition 3.2** (Worker).**

A worker $w$ of form <$${id,l}$$>, is a carrier of a mobile device who can accept spatial task assignments. The worker can be uniquely identified by his $id$ and his location is at $l$.

Intuitively, a worker is eligible to perform a task if his location is enclosed in the task region. In Figure 2(a), $w_{1}^{1}$ is eligible to perform $t_{1}^{1},t_{1}^{2}$ and $t_{1}^{3}$ while $w_{1}^{2}$ is qualified for $t_{1}^{1},t_{1}^{4},t_{1}^{5}$ and $t_{1}^{6}$. Furthermore, a worker’s report to one task can also be used for all other unexpired tasks whose task regions enclose the worker. As in Figure 2(b), $w_{2}^{1}$ is eligible to perform $t_{1}^{5}$ and $t_{1}^{6}$, which are deferred from time 1.

Let $W_{i}=\{w_{i}^{1},w_{i}^{2},...\}$ denotes the set of available workers at time $s_{i}$ and $T_{i}=\{t_{i}^{1},t_{i}^{2},...\}$ denotes the set of available tasks including tasks issued at time $s_{i}$ and previously issued un-expired tasks. Below we define the notions of worker-task coverage and coverage instance sets.

Definition 3.3** (Worker-Task Coverage).**

Given $w_{i}^{j}\in W_{i}$, let $C(w_{i}^{j})\subset T_{i}$ denotes the task coverage set of $w_{i}^{j}$, such that for every $t_{i}^{k}\in C(w^{j})$,

[TABLE]

We also say the worker $w_{i}^{j}$ covers the tasks $t_{i}^{k}\in C(w_{i}^{j})$. An example of a coverage in Figure 2(a) is $C(w_{1}^{1})=\{t_{1}^{1},t_{1}^{2},t_{1}^{3}\}$.

Definition 3.4** (Coverage Instance Set).**

At time $s_{i}$, the coverage instance set, denoted by $I_{i}$ is the set of worker-task coverage of form <$$w_{i}^{j},C(w_{i}^{j})$$> for all workers $w_{i}^{j}\in W_{i}$.

The coverage instance sets for the example in Figure 2 are illustrated in Table 1. For simplicity, we now assume the utility of a specific task assignment is binary within the task region and before the deadline. That is, assignment to any worker within a task region before the deadline has utility 1 (1 successful assignment), and 0 otherwise. As a result, task $t_{1}^{5}$ and $t_{1}^{6}$ being answered by worker $w_{1}^{2}$ at time $1$ is equivalent to it being answered by $w_{2}^{1}$ at time $2$.

Again, the goal of our study is to maximize task assignment given a budget, despite the dynamic arrivals of tasks and workers. Now, we formally define the notion of a budget.

Definition 3.5** (Budget).**

Budget $K$ is the maximum number of workers to select in a coverage instance set.

In practice, budget $K$ can capture the communication cost the SC-server incurs to push notifications to selected workers (Step 3 in Figure 1), or the rewards paid to the workers.

With these, we formally define the hyperlocal crowdsourcing problem with budget constraint as follows.

Definition 3.6** (Problem).**

Given a set of workers $W$, a set of available tasks $T$, a budget constraint $K$, and a utility function $U(\cdot)\in\mathbb{R}$, find a subset of workers $W^{\prime}$ of $W$ within the budget constraint, such that the total utility of the covered tasks is maximized.

3.1 Problem Taxonomy

3.1.1 Budget-per-time-period vs. Budget-per-campaign

In certain scenarios, the task requester may specify a budget constraint, i.e., the maximum number of workers to select, for each time period in a campaign, e.g., a day or a week. Given a set of time periods $\phi=\{s_{1},s_{2},...,s_{Q}\}$, a budget constraint $K_{i}$ is specified for each $s_{i}$. The challenge is to decide which workers to select within each time period. On the other hand, the task requester could specify a budget constraint for the entire campaign. Given a set of time periods $\phi=\{s_{1},s_{2},...,s_{Q}\}$ and assuming $L_{i}$ workers are selected for $s_{i}$, a budget constraint $K$ is specified for the sum of $L_{i}$’s. The new challenge of this problem variant is to allocate the total budget $K$ wisely over $Q$ time periods.

The choice of the constraint model depends on the financial flexibility of the task requester. Furthermore, the budget-per-time-period model is a special case of the budget-per-campaign model. As a result, the utility of budget-per-campaign solution is no worse than that of the budget-per-period solution for any problem instance.

3.1.2 Binary-utility vs. Distance-based-utility

Considering the utility of assigned tasks, our problem can be classified into binary-utility and distance-based-utility variants. In the binary-utility model, a task can be assigned to any worker located within the task radius to achieve utility $1$. Unassigned tasks will yield [math] utility. Therefore, the optimization objective is to maximize the total number of assigned tasks. However, for some applications, a worker who is closer to the task location may be “preferred” over other workers farther away. For example, in weather crowdsourcing applications, e.g., iRain [iRa (2016)], a closer worker can report more accurate rainfall data. The distance-based-utility model thus evaluates a task assignment to a specific worker with various distance functions.

3.1.3 Single-objective vs. Multi-objective

In the single-objective problem formulation, we aim to maximize the total utility of assigned tasks. On the other hand, crowdsourcing applications may have more than one, sometimes conflicting, objectives, to ensure long-term prosperity. For example, worker overloading can be a critical concern of the novel crowdsourcing platforms, in which only a few workers are frequently selected to optimize task assignments. Therefore, a multi-objective formulation can introduce a second objective to minimize the worker overloading phenomenon. The challenge is thus to find solutions considering the trade-off between the two objectives.

3.1.4 Offline vs. Online

Orthogonal to the dimensions above, our problem can be further classified into offline and online variants. The offline variant selects workers with complete knowledge of task/worker arrivals during the entire campaign. Although this is not practical, studying the offline variant allows us to eliminate the hardness arising from the randomness of the online problem and focus on the optimization in a deterministic setting. In the online variant, assignments have to be made in real-time for the currently arriving tasks/workers without complete knowledge of future arrivals. While it is more fitting for crowdsourcing applications, it is also intuitively more challenging — it is uncertain in nature when and where future tasks and workers may appear. Thus, effective worker selection must optimize the objective(s) in the long run.

The majority of this paper will focus on the online, binary-utility, single-objective problem with both per-time-period and per-campaign budget constraints. We will also show how to extend our solutions to the distance-based utility model as well as the multi-objective problem.

4 Hardness of the Problem

In this section we study the problem complexity of task assignment with budget constraint in hyperlocal spatial crowdsourcing. We show that two offline variants, i.e., budget-per-time-period vs. budget-per-campaign, of the problem are NP-hard and propose online heuristics in the next section.

4.1 Fixed Budget ${f\textit{MTC}}$

Problem 0** (Fixed-budget Maximum Task Coverage).**

Given a set of time periods $\phi=\{s_{1},s_{2},...,s_{Q}\}$ and a budget $K_{i}$ for each $s_{i}$, the fixed-budget maximum task coverage (fMTC) problem is to select a set of workers $L_{i}$ at every $s_{i}$, such that the total number of covered tasks $|\bigcup_{i=1}^{Q}\bigcup_{w_{i}^{j}\in L_{i}}C(w_{i}^{j})|$ is maximized and $|L_{i}|\leq K_{i}$.

This optimization problem is challenging since each worker is eligible for a subset of tasks. The fact that a task can be deferred to future time periods further adds to the complexity of the problem. With the following theorem, we proof that fMTC is NP-hard by a reduction from the maximum coverage with group budgets constraints problem (MCG) [Chekuri and Kumar (2004)]. MCG is motivated by the maximum coverage problem (MCP) [Feige (1998)]. Consider a given $I_{g}$, we are given the subsets $S=\{S_{1},S_{2},...S_{m}\}$ of a ground set $X$ and the disjoint sets $\{G_{1},G_{2},...,G_{l}\}$. Each $G_{i}$, namely a group, is a subset of $S=\{S_{1},S_{2},...S_{m}\}$. With MCG, we are given an integer $k$, and an integer bound $k_{i}$ for each group $G_{i}$. A solution to $I_{g}$ is a subset $H\subset S$ such that $|H|\leq k$ and $|H\cap G_{i}|\leq k_{i}$ for $1\leq i\leq l$. The objective is to find a solution such that the number of elements of $X$ covered by the sets in $H$ is maximized. MCP is the special case of MCG. Since MCP is known to be strongly NP-hard [Feige (1998)], by restriction, MCG is also NP-hard.

Theorem 1**.**

fMTC is NP-hard.

Proof 4.2**.**

We prove the theorem by a reduction from MCG [Chekuri and Kumar (2004)]. That is, given an instance of the MCG problem, denoted by $I_{g}$, there exists an instance of the MTC problem101010In this section, MTC refers to fixed-budget MTC for short, denoted by $I_{t}$, such that the solution to $I_{t}$ can be converted to the solution of $I_{g}$ in polynomial time. The reduction has two phases, transforming all workers/tasks across the entire campaign to a bipartite graph, and mapping from MCG to MTC. First, we layout the tasks and workers as two set of vertices in a bipartite graph in Figure 2(c). A worker $w_{i}^{j}$ can cover a task $t_{i}^{k}$ if both spatial and temporal constraints hold, i.e., Equations 2 and 1, respectively. In Figure 2(c), $w_{2}^{1}$ can cover $t_{1}^{4}$ and $t_{2}^{5}$, which are deferred from $s_{1}$ to $s_{2}$, represented by the dashed line.

Thereafter, MTC can be stated as follows. Selecting the maximum $K_{i}$ workers per group, each group represents a time period, such that the number of covered tasks is maximized (i.e., $|L_{i}|\leq K_{i}$). To reduce $I_{g}$ to $I_{t}$, we show a mapping from $I_{g}$ components to $I_{t}$ components. For every element in the ground set $X$ in $I_{g}$, we create a task $t_{i}^{j}$ ($1\leq j\leq|X|$). Also, for every set in $S$, we create a worker $w_{i}^{j}$ with $C(w_{i}^{j}=S_{j})$ ($1\leq j\leq m$). Consequently, to solve $I_{t}$, we need to find a subset $L_{i}\subset W_{i}$ workers of maximum size $K_{i}$ in each group whose coverage is maximized. Clearly, if an answer to $I_{t}$ is the set $L_{i}$ ($1\leq i\leq Q$), the answer to $I_{g}$ will be the set $H\subset S$ of maximum coverage such that $|H|\leq k=\sum_{i=1}^{Q}K_{i}$ and $|H\cap G_{i}|\leq k_{i}=K_{i}$ for $1\leq i\leq Q$.

As the transformation is bounded by the polynomial time to construct the bipartite graph, this completes the proof.

By a reduction from the MCG problem, we can now use any algorithm that computes MCG to solve the MTC problem. The greedy algorithm in [Chekuri and Kumar (2004)] provides $0.5$-approximation for MCG. For example, the greedy solution in Figure 2(c) is $\{w_{1}^{1},w_{2}^{1}\}$. However, the approximation ratio only holds in the offline scenario where the server knows $apriori$ the coverage instance set for every time period.

4.2 Dynamic Budget ${d\textit{MTC}}$

Problem 0** (Dynamic-budget Maximum Task Coverage).**

The dynamic-budget maximum task coverage problem (${d\textit{MTC}}$), is similar to ${f\textit{MTC}}$, except the total budget $K$ is specified for the entire campaign, i.e., $\sum_{i=1}^{Q}|L_{i}|\leq K$.

In the offline setting where the server is clairvoyant about the future workers and tasks, we prove the ${d\textit{MTC}}$ problem is NP-hard by reduction from the maximum coverage problem (MCP).

Theorem 2**.**

${d\textit{MTC}}$* is NP-hard.*

Proof 4.4**.**

We prove the theorem by a reduction from MCP. That is, given an instance of the MCP problem, denoted by $I_{m}$, there exists an instance of the MTC problem111111In this section, MTC refers to dynamic-budget MTC for short, denoted by $I_{t}$, such that the solution to $I_{t}$ can be converted to the solution of $I_{m}$ in polynomial time. The reduction includes two steps, transforming all workers/tasks across the entire campaign to a bipartite graph, and mapping from MCP to MTC. The first step is similar to that of Theorem 1, in which the workers and tasks from the entire campaign are transformed into a bipartite graph as illustrated in Figure 2(c). The mapping step can be considered as a special case of the proof in Theorem 1, in which there exists only one group of all budget. As the transformation is bounded by the polynomial time to construct the bipartite graph and MCP is strongly NP-hard, this completes the proof.

The results of these solutions to the offline scenarios will be used as the upper bounds of the results to the online solutions to be discussed in Section 5.

5 Online Task Assignment

In this section we focus on online variants: online fMTC when a budget constraint is given for each time period, and online dMTC, when the budget constraint is given for the entire campaign. We will introduce heuristics for each variant as follows.

5.1 Fixed Budget ${f\textit{MTC}}$

In the online scenario where workers and tasks arrive dynamically, it becomes more challenging to achieve the global optimal solution for Problem 4.1. Since the server does not have prior knowledge about future workers and tasks, it tries to optimize task assignment locally at every time period. However, the optimization within every time period, similar to the maximum coverage problem (MCP), is also NP-hard. A greedy algorithm [Feige (1998)] was proposed to achieve an approximation ratio of $0.63$, by choosing a set which contains the largest number of uncovered elements at each stage. This study shows that the greedy algorithm is the best-possible polynomial time approximation algorithm for MCP. Below we propose several greedy heuristics to solve the online ${f\textit{MTC}}$ problem, namely Basic, Spatial and Temporal.

5.1.1 Basic Heuristic

The Basic heuristic solves the online ${f\textit{MTC}}$ problem by using the greedy algorithm [Hochbaum (1996)] for every time period. At each stage, Basic selects the worker that covers the maximum number of uncovered tasks, depicted in Line 10 of Algorithm 1. For instance, in Figure 2(a), $w_{1}^{2}$ is selected at the first stage. At the beginning of each time period, Line 4 removes expired tasks from the previous time period. Line 5 adds unassigned, unexpired tasks to current task set. Line 12 outputs the covered tasks $C_{i}$ per time period which will be used as the main performance metric in Section 8. The algorithm terminates when either running out of budget or all the tasks are covered (Line 9).

Basic can achieve fast task assignment by simply counting the number of tasks covered by each worker (Line 10). However, it treats all tasks equally without considering the spatial and temporal information of each task, i.e., location and deadline. For example, a task located in a “worker-sparse” area may not be assigned in the future due to lack of nearby workers and thus should be assigned with higher priority at the current iteration. Similarly, tasks that are expiring soon should be assigned with higher priorities. Consequently, the priority of a worker is high if he covers a larger number of high priority tasks. Below we introduce two assignment heuristics that explicitly model the task priority given its spatial and temporal characteristics.

5.1.2 Temporal Heuristic

One approach to prioritizing tasks is by considering their temporal urgency. The intuition is that a task which is further away from its deadline is more likely to be covered in the future, and vice versa. As a result, near-deadline tasks should have higher priorities to be assigned than others. Consequently, a worker who covers a large number of soon-to-expire tasks should be preferred for selection. Based on the above intuition, we model the priority of a worker $w_{i}^{j}$ based on the remaining time of each task he covers as follows.

[TABLE]

The Temporal heuristic adapts Basic by selecting the worker with maximum priority at each stage. For instance, given two workers $w_{1}^{1}$ and $w_{1}^{2}$ at time $s_{1}$, where $C(w_{1}^{1})=\{t_{1}^{1},t_{1}^{2}\}$ and $C(w_{1}^{1})=\{t_{1}^{3}\}$. Suppose both $t_{1}^{1}$ and $t_{1}^{2}$ expire in $5$ time periods and $t_{1}^{3}$ expires in $2$ time periods. The Temporal heuristic chooses $w_{1}^{2}$ over $w_{1}^{1}$ as their priorities are $0.5$ and $0.4$, respectively. To implement Temporal, Line 10 in Algorithm 1 can be updated to select the worker with maximum priority defined as in Equation 3. We will empirically evaluate all heuristics in Section 8.

5.1.3 Spatial Heuristic

To maximize task assignment in the long term, we also consider the “popularity” of a task location as an indicator of whether the task can be assigned to future workers. Accordingly, we can spend the budget for the current time period to assign those tasks which can be only covered by existing workers. The “popularity” of a task region can be measured using Location Entropy [Cranshaw et al. (2010)], which captures the diversity of visits to that region. A region has a high entropy if many workers visit with equal probabilities. In contrast, a region has a low entropy if there are only a few workers visiting. We define the entropy of a given task as follows.

For task $t$, let $O_{t}$ be the set of visits to the task region $R(t.l,r)$. Let $W_{t}$ be the set of distinct workers that visited $R(t.l,r)$, and $O_{w,t}$ be the set of visits that worker $w$ made to $R(t.l,r)$. The probability that a random draw from $O_{t}$ belongs to $O_{w,t}$ is $P_{t}(w)=\frac{|O_{w,t}|}{|O_{t}|}$. The entropy of $t$ is computed as follows

[TABLE]

For efficient evaluation, $RE(t)$ can be approximated by aggregating the entropies of 2D grid cells within the task region $R(t.l,r)$ and the cell entropies can be precomputed using historical data. Since any worker located inside $R(t.l,r)$ can perform task $t$, $t$ is likely to be covered in the future as long as one grid cell inside $R(t.l,r)$ is “popular” among workers. Figure 3 illustrates the pre-computation of the entropy of task $t$. When a task arrives, we first identify the grid cell that encloses the task location, i.e., the white cell in the center, and slightly adjust the task region (solid circle) to be centered at the white cell (dashed circle). We approximate the task entropy by the entropy of the dashed circle, which can be computed. This is because the dashed circle is solely determined by the white cell and radius $r$. To further speed up the precomputation of all possible combination of the cell and the radius, we approximate the dashed circle by a set of cells whose centers are within the circle. With the entropy of every task covered by worker $w_{i}^{j}$, his priority can be calculated as follows

[TABLE]

Note that the constant $1$ is needed to avoid division by zero. Consequently, the Spatial heuristic greedily selects the worker with maximum priority at each stage. Line 10 in Algorithm 1 can be modified to reflect the spatial priority of each worker.

5.2 Dynamic Budget ${d\textit{MTC}}$

The second problem variant we study is more general, where a budget constraint is given for the entire campaign. This relaxation often results in higher task coverage. For example, in Figure 2, if budget $1$ is given at every time period, we select $w_{1}^{1}$ and $w_{2}^{1}$ and obtain the coverage of 5. However, the dynamic-budget variant yields higher coverage of 6 by selecting $w_{1}^{1}$ and $w_{1}^{2}$ at time $1$. Below we study the problem complexity in the offline scenario and propose adaptive budget allocation strategies for the online scenario.

The challenge of the online $d{\textit{MTC}}$ problem is to allocate the overall budget $K$ over $Q$ time periods ($K\geq Q$) optimally, despite the dynamic arrivals of workers and tasks. Below we introduce several budget allocation strategies. Once a budget is allocated to a particular time period, we can adopt previously proposed heuristics, i.e., Basic, Spatial, Temporal, to select the best worker.

The simplest strategy, namely Equal, equally divides $K$ to $Q$ time periods; each time period has $K/Q$ budget and the last time period obtains the remainder. However, Equal may over-allocate budget to the time periods with small numbers of tasks. Another strategy is to allocate a budget to each time period proportional to the number of available tasks at that time period, i.e., $\frac{|T_{i}|}{|T|}K$, where $|T|$ is the total number of tasks. However, $|T|$ is not known a priori. Furthermore, we may still over-allocate budget to any time period with large $|T_{i}|$, if none of the tasks can be covered by any workers (or all the tasks can be covered by 1 worker). We cannot allocate budget optimally without looking at the coverage instance set at each time period.

5.2.1 Adaptive Budget Allocation

To maximize task assignment, we need to adaptively allocate the overall budget and consider the “return” of selecting every worker, i.e., the worker priority, given the dynamic coverage instance set at every time period. We define the following two notions. Delta budget, denoted as $\delta_{K}$, captures the current status of budget utilization, compared to a baseline budget strategy $\{K^{base}[t],t=1,\dots,Q\}$, e.g., the Equal strategy . Given a certain baseline $\{K^{base}[t]\}$, $\delta_{K}$ is the difference between the cumulative baseline budget and the actual budget spent up to time period $s_{i}$. Formally, at any time period $s_{i}$,

[TABLE]

A positive $\delta_{K}$ indicates budget is under-utilized, and vice versa. Another notion is delta gain, denoted as $\delta_{\lambda}$, which represents the return of a worker currently being considered ($\lambda_{l}$) compared to the ones selected in the past ($\overline{\lambda_{l-1}}$). Formally,

[TABLE]

where $\lambda_{l}$ is the gain of the current worker, calculated by any previously proposed local heuristic, i.e., as $|priority(w_{i}^{j})|$. $\overline{\lambda_{l-1}}$ is the average gain of previously added workers, i.e., $\overline{\lambda_{l-1}}=\frac{1}{l-1}\sum_{t=1}^{l-1}\lambda_{t}$. A positive $\delta_{\lambda}$ indicates the current worker has higher priority than the historical average, and vice versa.

Based on the contextual information $\delta_{K}$ and $\delta_{\lambda}$ at each stage of worker selection, we examine all available workers at the currently time period and decide whether to allocate budget $1$ to selecting any worker. Intuitively, when both $\delta_{K}$ and $\delta_{\lambda}$ are positive, i.e., the budget is under-utilized and a worker has higher priority, the selection of the considered worker is favored. When both are negative, it may not be worthwhile to spend the budget. The other cases when one is positive and the other is negative are more complex, as we would like to spend budget on workers with higher priority but also need to save budget for future time periods in case better worker candidates arrive.

Our solution to the sequential decision problem is inspired by the well-known multi-armed bandit problem (MAB), which has been widely studied and applied to decisions in clinical trials, online news recommendation, and portfolio design. $\epsilon$-greedy, which achieves a trade-off between exploitation and exploration, proves to be often hard to beat by other MAB algorithms [Vermorel and Mohri (2005)]. Hence, we propose an adaptive budget allocation strategy, based on contextual $\epsilon$-greedy algorithm [Li et al. (2010)]. We illustrate our solution in Figure 4.

At each stage of the local heuristic, a binary decision to make is whether to allocate budget $1$ to activate the current worker with the highest priority. The contextual $\epsilon$-greedy algorithm allows us to specify an exploration-exploitation ratio, i.e., $\epsilon$, based on the worker’s context, i.e., $\delta_{K}$ and $\delta_{\lambda}$. As depicted in Figure 4, an $\epsilon_{i}$-greedy algorithm is used to determine whether to select the current worker based on his $\delta_{K}$ and $\delta_{\lambda}$. For each case, a YES decision is made with $1-\epsilon_{i}$ probability and a NO decision with $\epsilon_{i}$ probability. By default, we set $\epsilon_{1}=1$ and $\epsilon_{4}=0$ to reflect NO and YES decisions, respectively, as discussed before. When $\delta_{K}$ and $\delta_{\lambda}$ have different signs, the decision is not as straightforward as the other cases and thus we set $\epsilon_{2}=\epsilon_{3}=0.5$ to allow YES and NO decisions with equal probabilities. The pseudo code of our adaptive algorithm is depicted in Algorithm 2.

5.2.2 Historical Workload

Previously our solution is simplified by considering $\{K^{equal}[t]\}$ as the baseline budget strategy. Since human activity exhibits temporal patterns, understanding those patterns may help to guide budget allocation. Therefore, we propose to compute a baseline budget strategy with historical data that captures the expected worker/task patterns. The study in [Musthag and Ganesan (2013)] shows the time-of-day usage patterns of workers in mobile crowdsourcing applications. The activity peaks are between 4 to 7 pm when workers leave their day jobs. Similar patterns are observed in Foursquare and Gowalla data sets in Figure 5. Figure 5(a) shows the hourly count of check-ins present three peaks, i.e., during lunch and morning/afternoon commute. In Figure  5(b), we can observe peak check-in activities during weekends.

With historical worker and task information, we can leverage the optimal budget allocation strategy in the recent past and use it as the baseline strategy in Equation 6. We propose to learn the budget allocation of previous time periods, namely workload, using the greedy algorithm for the offline $d\textit{MTC}$ problem. To guide future budget allocation decisions, the previous workload $K^{prev}[]$ will be used as the baseline in Equation 6. We will empirically evaluate our proposed solutions in the experiment section.

6 Worker Overload

In this section, we present an enhancement to our solution in order to avoid repetitive activations of the same workers. The practical implication is that those workers who locate in popular areas can be repeatedly selected by our heuristics. Overloading workers may result in undesirable consequences, such as tasks being rejected and the workers either feel annoyed or stressed out to report. Several recent studies [Alfarrarjeh et al. (2015), Zhang et al. (2016), Liu et al. (2016a)] also discuss the issue of over-assigning tasks to workers. These studies minimize worker overloading by balancing the workload of the workers. For example, the objective is to find an assignment that minimizes the variance of the workload among workers, i.e., maximize the so-called social fairness [Liu et al. (2016a)]. Another work [Alfarrarjeh et al. (2015)] also aims to assign a similar number of tasks to each worker. However, none of these studies considers task assignment and worker overloading as a multi-objective optimization problem.

Our idea is to minimize the phenomenon of overloading. This requires to maintain the number of times each worker $w_{id}$ has been activated up to time $i$, map$$<$$w_{id},count_{i}(w_{id})$$>. The counter is defined as:

[TABLE]

where $d(w_{k}^{j})$ represents a decision to select the $j^{th}$ worker at time $k$: $d(w_{k}^{j})=1$ if the worker is selected, otherwise $d(w_{k}^{j})=0$. The brackets enclose a condition that includes the term $d(w_{k}^{j})$ to the sum iff $w_{k}^{j}$ is identified by the same $id$.

We include minimization of worker overloading as another objective to coverage maximization. In the following, we formulate a multi-objective optimization (MOO) problem and propose solutions in both fixed-budget and dynamic-budget scenarios.

6.1 Fixed Budget $f\textit{MTC}$

In the fixed-budget setting, we formally define the multi-objective optimization (MOO) problem for each time instance $i$ below:

[TABLE]

[TABLE]

Equation 9a maximizes the coverage of the selected workers while Equation 9b minimizes the highest activation count across all workers present at the time. The constraint in Equation 9c ensures the number of selected workers does not exceed the budget $K_{i}$ at each time instance.

Rather than coming up with heuristics to sort the workers according to two objectives, we adopt a widely used approach, i.e., nondominated sorting genetic algorithm (NSGA) [Srinivas and Deb (1994)], to solve the MOO formulation for each time instance. Intuitively, nondominated sorting is to maintain stable nondominated fronts (i.e., subpopulations of good individuals) in a multi-dimensional space, where each dimension corresponds to an objective. A nondominated front, also referred to as Pareto optimal, is a solution where none of the objective functions can be improved in value without degrading other objective values. The advantage of genetic algorithms is that they simultaneously deal with a set of possible solutions i.e., population, which enables us to find several members of the Pareto optimal set in a single run of the algorithm. We outline our solution based on the NSGA algorithm in Algorithm 3.

The results of NSGA, at the end of while loop, include a set of nondominated fronts. Subsequently, in Line 7 we select the best individual solution based on a weighted sum of objective values:

[TABLE]

In Equation 10, $\alpha$ is a linear coefficient, $0<\alpha<1$, to specify the weight for each objective. The higher $\alpha$, the more important the objective of the Equation 9a in comparison to that of Equation 9b. The minus sign indicates the minimization objective in Equation 9b. Both objective functions are normalized by the total number of tasks $|T_{i}|$ and the total number of time instances $Q$, respectively. In our experiments, we adopted NSGA-II version [Deb et al. (2002)] for implementing Algorithm 3 and set $maxgen=50,000$.

6.2 Dynamic Budget $d\textit{MTC}$

In the dynamic-budget setting, the multi-objective optimization (MOO) formulation is similar to Equation 9a, 9b but the total budget is constrained over all time periods. Therefore, the constraint 9c is replaced by the following constraint:

[TABLE]

[TABLE]

In the online setting, we need to simultaneously consider the task coverage and the number of activations of the candidate worker, in order to optimize both objectives. As a result, we modify the adaptive strategy in Section 5.2.1 and define the gain $\lambda_{l}$ of the current worker $w^{j}_{i}$ in (7) to be a linear combination of the number previous activations and his priority:

[TABLE]

In equation 12, $priority(w_{i}^{j})$ and $count(w_{i}^{j})$ are respectively the priority of the worker calculated by any previously proposed local heuristic, and the number of times that worker was selected. The coefficient $\alpha$ can be varied to balance the importance of the overloading and the priority.

7 Distance-based Task Utility

Thus far, our goal is to maximize the number of assigned tasks, assuming assigning to any worker within the task region is equivalent. However, in practice an assignment of a task to a nearby worker may yield higher utility than that of a farther worker [Miao et al. (2016)]. Thus, in this section we aim to generalize the binary-utility model to a distance-based-utility variant, i.e., maximizing the utility of covered tasks. We assume the utility of worker $w$’s response to task $t$ is a function of the spatial distance between them: $utility(t,w)=f(dist(t,w))$. And $f$ is a decreasing function of worker-task distance. Intuitively, the utility is at the highest when the worker is co-located with the task and decreases as the worker-task distance increases. The utility is zero if the distance is larger than task radius. We consider three cases depicted in Figure 6: (i) Binary, where utility has value 1/0 (ii) Linear, where utility decreases linearly with the worker-task distance and (iii) Zipf, where utility follows Zipfian distribution with skewness parameter $s$. The higher the value of $s$, the faster utility drops.

This extension can be incorporated into all the previously developed algorithms. Specifically, Algorithm 1 (Line 10) now chooses the worker that maximizes utility increase at each stage.

[TABLE]

With the temporal heuristic, Equation 3 becomes:

[TABLE]

In the same fashion, with the spatial heuristic, Equation 5 becomes:

[TABLE]

With the adaptive budget strategies, the gain of a candidate worker is adapted similarly.

8 Performance Evaluation

8.1 Experimental Methodology

We adopted real-world datasets from location-based applications, summarized in Table 2, to emulate spatial crowdsourcing (SC) workers and tasks. We consider Gowalla, Foursquare users as SC workers and the venues as tasks. The Gowalla dataset contains check-ins for 224 days in 2010, including more than 100,000 spots (e.g., restaurants), within the state of California. By considering each day as a unit time period, all the users who checked in during a day are available workers for that time period in our setting. The Foursquare dataset contains the check-in history of 45,138 users to 89,968 venues over 384 hours in Pittsburgh, Pennsylvania. We considered each hour as a unit time period for this dataset.

We generated a range of datasets with SCAWG toolbox [To et al. (2016a)] by utilizing real-world worker/task spatial distributions and varying their arrival rate. We generated worker count following COSINE (default) and POISSON distributions with mean = 50 and set default value of task count per time period to be constant, i.e., 1000. We denote Go-POISSON a dataset that uses Gowalla for the spatial distribution and POISSON for the worker arrival rate.

In all of our experiments, for Gowalla dataset, we varied the total number of time periods $Q\in$ {7, 14, 28, 56} and the budget $K\in$ {56, 112, 224, 448, 896, 1288}. For Foursquare, $Q\in$ {24, 48, 72, 96} and $K\in$ {24, 48, 96, 192,…, 1536} because we modeled a time period as one hour. The task duration $\delta$ was randomly chosen from 1 to $Q$ and the task radius $r\in$ {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} km. The choices of $r$ and $\delta$ values are defined by the CHRS experts. Default values are shown in boldface. Finally, experiments were run on an Intel(R) Core(TM)i7-2600 CPU @ 3.40 GHz with 8 GB of RAM.

8.2 Experimental Results

In the following we evaluate our solutions in terms of the number of covered tasks, i.e., task coverage. We first show the performance of offline solutions (Section 8.2.1). Then we present the results for the online scenario, including local heuristics, adaptive budget strategy, and workload heuristic (Section 8.2.2). We next show the results of distance-based utility and worker overloading (Section 8.2.3), followed by runtime measurements (Section 8.2.4).

8.2.1 Optimal Solutions for Offline Setting

We implemented the offline solutions for the two problem variants, $f{\textit{MTC}}$ and $d{\textit{MTC}}$ (Section 4), using integer linear programming. These algorithms provide optimal results, which are used as the upper-bounds of the online algorithms. Figure 7(a) illustrates the results for Go-POISSON by varying the total budget $K$. As expected, higher budget yields higher coverage. Also, DynamicOff yields higher coverage than FixedOff as FixedOff is the special case of DynamicOff. However, the higher the budget, the smaller the performance gap between them. This effect can be explained by the diminishing return property of the max cover problem. That is, the more workers are selected, the smaller gain of each selected worker. We also evaluated the offline solutions by varying task radius $r$ (Figure 7(b)). Intuitively, when $r$ increases, more workers are located within the task’s spatial range. This means that a worker is eligible to perform more tasks, which yields higher task coverage.

Similar trends were observed for Go-POISSON as shown in Figures 7(c) and 7(d). We observe a small difference between FixedOff and DynamicOff for Go-POISSON in Figure 7(d). However, when the arrival rate has high variance, such as in Go-COSINE, DynamicOff shows more coverage over FixedOff in Figure 7(b). The reason is that FixedOff uses a constant budget to the time periods with high spikes while DynamicOff can allocate more budget to those time periods to cover more tasks.

8.2.2 Local Heuristics and Budget Allocations for Online Setting

The Performance of Heuristics: We evaluate the performance of the online heuristics for $f{\textit{MTC}}$ from Section 5.1, Basic, Spatial and Temporal. Figures 8(a) shows the improvements of Spatial and Temporal over Basic on Go-COSINE. When the budget is high, we observe that the simple heuristic Basic already obtains results close to the optimal solution. This is because most workers are selected. Furthermore, Spatial and Temporal yield 2% and 4% higher coverage than Basic (at $K=224$) and their performance converges as $K$ increases. Similar trends can be observed when increasing the task radius $r$ in Figure 8(b).

Adaptive Budget Allocation Strategy: We evaluate the performance of the adaptive budget allocation strategy in Section 5.2.1 by comparing with three baseline strategies inspired by a few previous studies, namely, Equal [To et al. (2016b), Tran-Thanh et al. (2013)], Random [Tran-Thanh et al. (2013)] and Naive [Kazemi et al. (2013), Ji et al. (2016), Zhang et al. (2015)]. In Equal budget strategy, the total budget is allocated equally to the time intervals. In Random budget strategy, a random positive number $k_{i}$ is generated for each time interval $i$ and then each time interval $i$ is given a budget $K_{i}=K*\frac{k_{i}}{\sum_{1}^{Q}k_{i}}$. In Naive budget strategy, there is no particular limitation for the budget of each time interval, the budget is used until no more worker is available or entire budget is exhausted. We use local heuristic Basic to compare the performances of budget strategies. Figure 9 shows the results of the budget strategies when varying the total budget $K$. AdaptB is shown to be the best in coverage. EqualB and RandomB do not perform as well as AdaptB, as they lack an intelligent budget allocation strategy. NaiveB performs poorly as it selects the workers on a first-come-first-serve basis without considering future time intervals. As the result, the total budget is quickly exhausted during the first few time intervals. The difference between AdaptB and the others is higher with GO-COSINE because it has more fluctuation in the number of workers over time. This shows that AdaptB can quickly adapt to the dynamic arrivals of workers.

Historical Workload Improvement: We also evaluate the performance of the adaptive budget allocation strategy applied with local heuristics, in which Temporal is shown to perform better than Basic. Figure 10 shows the results of EqualB, AdaptB, AdaptT and AdaptTW (AdaptT with historical workload improvement) when varying total budget $K$ and task radius $r$ on Go-COSINE and Go-POISSON datasets. We include DynamicOff as the optimal result for reference. As can be seen in that figure, with small budgets, AdaptTW, which uses historical optimal workload as the baseline budget strategy, has higher coverage than the others and with $K=56$, EqualB performs better than AdaptB, AdaptT. The reason is with small budgets, AdaptB and AdaptT do not have enough contextual information. With higher budget (K $\geq 448$), the adaptive algorithms perform better than EqualB and their results are close to the optimal result. We observe similar results when varying $r$.

We further study the performance of various budget allocation strategies by plotting the task coverage across multiple workloads using boxplots. Figure 11 shows the results of the techniques with the default parameter setting. As can be seen in the figure, the adaptive algorithms perform better than EqualB especially with GO-COSINE dataset. While AdaptTW has the highest median, minimum, maximum values, AdaptT is the most stable method with the smallest difference between the minimum and the maximum values.

8.2.3 Distance-based Task Utility and Worker Overload

Distance-based Task Utility: We show the performance of AdaptT under distance-based functions for task utility from Section 7. In Figure 12(a), we observe that the obtained task coverage in the cases of Linear and Zipfian are similar to the result in the binary-utility model. We also present the overlapping ratio of selected workers between distance-based utility and binary utility as shown in Figure 12(b). The figure shows that when total budget increases, initially, the ratio decreases because of more different workers can be selected and then ratio increases because when the budget is large enough, most workers are selected in both cases, resulting in large overlaps.

Worker Overload Minimization: In this section we evaluate the performance of the multi-objective optimization techniques in both fixed-budget and dynamic-budget scenarios. The techniques are evaluated in terms of balancing the trade-off between maximizing task coverage and minimizing worker overload. In the fixed-budget scenario (Section 6.1), EqualGA refers to the equal-budget strategy with NSGA in Algorithm 3. We observe that varying coefficient $\alpha$ does not significantly change task coverage (Figure 13(a)). This is due to the equal allocation of the total budget to each time period, which yields suboptimal task coverage. By increasing $\alpha$ the average number of activations per worker in Figure 13(b) shows a slightly decreasing trend, due to a higher weight on the second objective. In the dynamic-budget scenario, AdaptT-MOO refers to adaptive budget allocation with temporal local heuristic and the multi-objective optimization in Section 6.2. Figure 13(c) shows that the task coverage is quite stable as $\alpha$ increases while the average number of activation decreases significantly. This means that our adaptive budget allocation strategy achieves workload balancing among the workers at a very small cost in utility. Furthermore, without loss of generality, based on the observations, we set $\alpha=0.1$ for the following experiments.

Figure 14 shows the distribution of activation counts of selected workers when $K=448$. In the fixed-budget setting, it is shown that EqualGA does not cover as many tasks as EqualB but it activates more workers for a small number of times, i.e., 1, 2, and 3 times. In the dynamic-budget setting, AdaptT-MOO also has more workers with a small number of activations and yields comparable task coverage, compared to AdaptT. We conclude that our solutions can mitigate worker overloading without compromising task assignment.

8.2.4 Runtime Measurements

Figure 15 shows the runtime performance of our online algorithms by varying the number of tasks per time period. As expected, the runtime linearly increases when the number of tasks grows. In the fixed-budget scenario, the runtimes of the local heuristics (e.g., Temporal) are the same as Basic while the runtime of EqualGA is higher due to having a large number of iterations for Algorithm 3. We do not show the runtime of Spatial heuristic and Zipfian utility model but their runtimes are similar to EqualB and EqualT-Linear, respectively. In the dynamic-budget scenario, the runtime of AdaptTW is higher than AdaptT, AdaptT-Linear, and AdaptT-MOO. This suggests that the workload heuristic significantly increases the overhead of AdaptT. However, the MOO extension does not incur observable runtime overhead in the dynamic-budget scenario.

9 Discussion

Existing studies show that knowing the worker mobility pattern a priori can improve the efficiency of the task assignment [Ji et al. (2016), Zhang et al. (2015)]. Even though, our solution does not consider individual worker mobility pattern, i.e. the worker’s trajectory, for task assignment. However, our heuristics (Section 5.1.3) does consider worker population mobility pattern by prioritizing tasks whose locations are not likely to be visited by many workers in the future. Furthermore, our dynamic budget algorithm (Algorithm 2) takes into account the dynamic arrivals of workers and tasks as well as their co-location relationship.

It is worth noting that in our problem settings 1) task assignment is real-time and online and 2) workers are not required to travel to perform tasks. Workers can respond to a task immediately after receiving the task notifications from SC-server. Therefore, they do not need to perform a sequence of tasks as in a typical mobile crowdsourcing where workers often chain multiple tasks to maximize their earnings while minimizing travel time [Ji et al. (2016)]. In addition, workers’ trajectories within the same task region would not have much impact in our problem setting, as the workers are not required to travel to perform the task. Obviously, as the workers move, they may become relevant to another spatial task and/or irrelevant to the prior task, which can be represented as the addition and deletion of a worker in our framework at a given snapshot.

10 Conclusion

Motivated by weather crowdsourcing applications, we introduced the problem of Hyperlocal Spatial Crowdsourcing, where tasks can be performed by workers within their spatiotemporal vicinity. We studied task assignment in Hyperlocal SC to maximize the covered tasks without exceeding the budget for activating workers. A range of problem variants was considered, including offline vs. online, budget constraint for each time period vs. for the entire campaign, single objective vs. multiple objectives, and binary vs. distance-based utility. We showed that the offline variants are NP-hard and proposed several local heuristics and the dynamic budget allocation for the online scenario which utilize the spatial and temporal properties of workers/tasks. We generated spatial crowdsourcing workloads with SCAWG tool and conducted extensive experiments. We concluded that AdaptT, which merits the temporal local heuristic and dynamic budget allocation, is the superior technique in terms of utility and runtime. The extensions to measure distance-based utility and to minimize worker overloading were shown to be very effective and do not impose significant runtime overhead. As future work, we will consider non-uniform activation cost of the workers, which represents the reputation or the compensation demand of each worker. We will also consider assigning a task to multiple workers to improve the quality of collected data and utilizing known worker mobility patterns to boost task assignment.

11 Acknowledgments

We would like to thank CHRS researchers, especially Dr. Phu Dinh Nguyen for leading the development of the iRain project: http://irain.eng.uci.edu/.

This research has been funded by NSF grants IIS-1320149, CNS-1461963, the USC Integrated Media Systems Center, and the University at Albany. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of any of the sponsors such as NSF.