Agora: A Unified Asset Ecosystem Going Beyond Marketplaces and Cloud Services

TL;DR

Agora introduces a decentralized, open ecosystem that enables flexible exchange and combination of data, algorithms, and infrastructure assets, reducing lock-in effects and entry barriers in AI and data science.

Contribution

The paper presents Agora, a novel unified asset ecosystem that integrates data, algorithms, and infrastructure in a decentralized environment, facilitating flexible asset exchange and new application development.

Findings

Supports dynamic creation and modification of assets

Enables combining assets for novel applications

Reduces lock-in effects and entry barriers

Abstract

Data, algorithms, and compute/storage infrastructure are key assets that drive data science and artificial intelligence applications. As providing all these assets requires a huge investment, data science and artificial intelligence technologies are currently dominated by a small number of providers who can afford these investments. This leads to lock-in effects and hinders features that require a flexible exchange of assets among users. In this vision paper, we present Agora, a unified asset ecosystem. The Agora system provides the technical infrastructure that allows for offering and using data and algorithms, as well as physical infrastructure components. Agora is designed as an open ecosystem of asset marketplaces and provides to a broad audience not only data but the entire data value chain (including computational resources and human expertise). Agora (i) leverages a fine-grained exchange of assets, (ii) allows for combining assets to novel applications, and (iii) flexibly executes such applications on available resources. As a result, Agora overcomes lock-in effects and removes entry barriers for new asset providers. In contrast to existing data management systems, Agora operates in a heavily decentralized and dynamic environment: Data, algorithms, and even compute resources are dynamically created, modified, and removed by different stakeholders. Agora presents novel research directions for the data management community as a whole: It requires to combine our traditional expertise in scalable data processing and management with infrastructure provisioning as well as economic and application aspects of data, algorithms, and infrastructure.