Data Markets to support AI for All: Pricing, Valuation and Governance

TL;DR

This paper explores a data market framework that assesses intrinsic and extrinsic data values, providing valuation methods for data in various contexts to facilitate equitable AI development.

Contribution

It introduces a comprehensive data valuation approach considering relevance, uniqueness, and market dynamics, enhancing data trading and governance for AI.

Findings

Proposes valuation techniques based on relevance and uniqueness.

Defines intrinsic and extrinsic data value metrics.

Offers methods for absolute, relative, and conditional data valuation.

Abstract

We discuss a data market technique based on intrinsic (relevance and uniqueness) as well as extrinsic value (influenced by supply and demand) of data. For intrinsic value, we explain how to perform valuation of data in absolute terms (i.e just by itself), or relatively (i.e in comparison to multiple datasets) or in conditional terms (i.e valuating new data given currently existing data).

Full text

Data Markets to support AI for All: Pricing, Valuation and Governance

Ramesh Raskar, Praneeth Vepakomma, Tristan Swedish, Aalekh Sharan*

Massachusetts Institute of Technology

Cambridge, MA 02139, USA

{raskar,vepakom,tswedish}@mit.edu

1 Introduction

We discuss a data market technique based on intrinsic (relevance and uniqueness) as well as extrinsic value (influenced by supply and demand) of data. For intrinsic value, we explain how to perform valuation of data in absolute terms (i.e just by itself), or relatively (i.e in comparison to multiple datasets) or in conditional terms (i.e valuating new data given currently existing data).

2 Motivation for creating Data Markets

AI will benefit from Liquid Markets: Data is increasingly concentrated in large firms. For startups, and small organizations it is increasingly difficult to compete as the lack of availability of data can stymie any and all efforts to build better machine learning algorithms. Algorithmic capability indeed increases with the availability and quality of data. One way to tackle this is the marketplace approach. By creating conditions such that data, the raw material for AI, can be bought and sold with security, privacy and consent safeguarded, specialized niches will be created and firms will be able to tackle a smaller subset of the problem. A precondition for this large scale collaboration to occur is the existence of liquid markets at various steps of the value chain.

Example: Consider a diagnostic healthcare company aiming to do an acquisition of labeled X-ray images from various hospitals for developing state of the art diagnostics. The key problem in such a setting is: “How can value of datasets from each hospital be estimated to decide their price?” The data for some hospitals can belong to unique health traits and demographics and can be very valuable for the diagnostic use-case of the company while data from some other hospitals may be of a relatively much lower value.

A key problem therefore is that obtaining large amounts of diverse, yet useful data costs a lot of resources. There are also diminishing returns at some point when additional data does not improve the capabilities of the algorithm, if additional data is not acquired intelligently in a cost-effective manner.

3 Data Valuation for AI

Absolute, relative or conditional data purchase: Another required facet to setting up a data market is to build a capability to perform valuation of data in absolute terms (i.e just by itself), or relatively (i.e in comparison to multiple datasets) or in conditional terms (i.e valuating new data given currently existing data).

Intrinsic or extrinsic data valuation: Any of these data valuation use-cases can be performed via intrinsic factors of evaluation such as based on quality of information within the dataset or via extrinsic factors of evaluation such as based on demand-supply, market economics, game theoretic mechanisms and speculative market forces or via a combination of both as in [8, 9, 1, 3, 10, 17, 18, 16].

Goal dependent or independent data trading: An additional slicing to this problem includes goal specific or goal independent data valuation depending on whether there is a specific well-defined goal for the data purchase or if it is exploratory by design for a goal that is currently undefined; but would be drafted later on.

Horizontal or vertical data acquisition: In addition to all these situations of data valuation, yet another categorization is based on whether the data acuiqition is being done vertically (in terms of acquiring attributes/columns) or horizontally (acquiring records/rows) as in[5, 7]. This terminology of ’vertical partitioning’ and ’horizontal partitioning’ extends from the databases as well as distributed systems research communities.

Privacy aware data valuation: Ideally such a data valuation needs to be acheived by looking at as few records per data source as possible or via privacy aware AI [15, 14, 13, 6]. Pooling of all data at a centralized location defeats the central purpose and the data sharing constraints of privacy, security, safety, fairness and resource efficiency need to be kept in consideration with regards to a data valuation solution for data markets.

Relevance and diversity of data acquisition: An optimal data purchase under these constraints needs to cater to high utility and low redundancy (high diversity) of data in terms of incremental benefit obtained. There is often a tradeoff of utility vs. diversity of data that needs to be considered in realistic settings. This concept has been the guding principle for techniques like sure independence screening (SIS) and conditional sure independence screening [4, 19, 2] currently actively being studied in the field of statistics and in min redundancy max relevance (mRMR) [11] in the field of data mining, during the precursory periods of current day AI and machine learning.

As shown in Figure 1, a robust data valuation acts as a good input for data pricing as well as for building an optimal market basket of data for every data consumer.

The intent of sharing these possibilities is to motivate further discussion and research. We summarize some of these points with regards to data valuation in the context of data markets as shown below:

4 Data is very complex to price

The value of an incremental unit of data is also conditionally dependent on data already possessed by the prospective data buyer entity that is valuating it. This is because one would like to obtain relevant yet diverse data from what is already available in-house. In addition, data can be acquired for performing either a similar or a more diverse task in comparison to the current use-cases being applied on data that is already available in-house. Also, there are so many archetypes that it is difficult to find a proxy variable (like weight or number in the case of other goods) that can be used to define the data. Since seamless discovery and a small spread in price is essential for a marketplace to function well, it has been challenging thus far to create a functioning data marketplace. A thorough data pricing startegy needs to adhere to the following guiding principles.

Data Pricing Guidelines

Liquidity: models freshness of dat in terms of value vs diminished/increased value over time 2. 2.

Traceability: can be only ‘sold’ once, or sold non-exclusively 3. 3.

Consent: maintains privacy of owner, tracks consent over time, and reduces friction with smart contracts or data concierges. 4. 4.

Neutrality: accessible to all buyers to prevent unfair trading practices. Otherwise, it would encourage some players (be it large or small) to unfairly price out the rest of prospective buyers during the trading. 5. 5.

Recourse: Allows for calling back, provides right to be forgotten, allows for some course correction, broadly remains self-sustaining.

5 Data sharing challenges that data markets need to address

Although acquiring the right amount of quality data is ideal, data sharing is heavily impeded by friction caused by lack of trust, data sharing regulations such as HIPAA/GDPR, lack of ease and lack of incentive. We further expand on these factors that cause data friction.

Lack of incentives:

1. (a)

Large organizations need incentive mechanisms to share data with small players. For example, an incentive for data sharing between large centralized hospitals and local clinics, testing centers could be to foster better provision of health. 2. (b)

Big tech players have taken a lead and are rapidly collecting and hoarding data while monopolizing the data resources and are preventing small players from entering into data acquisition. This stifles innovation. 3. (c)

Individuals need incentives to share their data as they happen to generate and own tremendous amount of data on a daily basis. But this leads to the burden of consent management which is too complex to manage granularly across different modalities, time horizons, and trust-levels in data buyers. 4. (d)

Governments and non-profit are often not allowed to sell data for monetary gains . 2. 2.

Lack of ease of sharing data: Due to lack of automated processes, digitization, access to data pre-processing pipelines, compatible data schemas, lack of standardization across data sources and other forms of siloing of socially beneficial data; seamless data sharing is restricted. To summarize, these factors include:

1. (a)

Lack of digitization and lack of use cases 2. (b)

Lack of data standardization across multiple sources 3. (c)

Collection of data currently will likely cost more than market price of data 4. (d)

Socially beneficial good data is locked away (e.g. with government, non-profits, hospitals, remote sensing data) 3. 3.

Lack of trust: Data sharing can also be impeded by the factors of market forces, need for maintaining trade secrets, competitive economy that impedes trust, fear of losing control and accountability over future usage of data for adversarial purposes. To summarize, these factors include cases when:

1. (a)

Data owner does not trust what the buyer will do with data in a competing environment 2. (b)

Data indirectly contains trade secrets of the data owner 3. (c)

Fear of adversarial future usage of shared data 4. 4.

Regulations: Data sharing is regulated for privacy, security, fairness and safety and therefore any data transactions for performing basic data analysis or for any advanced AI/ML usecases has to be aware of these constraints and be able to safely circumvent these friction points while also maintaining compliance with the law. To summarize:

1. (a)

In sectors such as health, finance and cybersecurity that are tightly governed by local, federal and international data sharing regulations such as HIPAA, GDPR, COX, PCI, SHIELD, we need a new strategy for safe data sharing. 2. (b)

Policies for inter as well as intra organizational data sharing have to be adhered to. 3. (c)

The origination of data may have country specific regulations on usage. Therefore international regulations need to be adhered to with respect to both the data provider as well as the data consumer. 4. (d)

There are policies where data cannot physically leave the premises of the data owners.

6 Governance and encouragement for a data market ecosystem

In addition from the perspective of governance, the following would be key to support the setup as well as to sustain a good ecosystem for data markets [12].

1. (a)

Need to support technological solution vs market solution vs policy-driven solutions. 2. (b)

Data governance policies by undertaking a study of changes needed in existing legal/regulatory frameworks. 3. (c)

Standardization of data sharing 4. (d)

Setting up national ‘nodes’ of servers for data exchange (like stock exchanges) 5. (e)

‘Clean Data’ credits like ‘clear air’ carbon credits 6. (f)

Treat data as labor (it’s from activity, that creates value) 7. (g)

Ethics and bias: self certification as well as audits