Improving Web API Usage Logging

TL;DR

This paper explores the potential of WAPI usage logs for analysis, highlights challenges caused by log format issues, and proposes solutions to improve log quality for better consumer behavior insights.

Contribution

It distinguishes between development and production logs, analyzes their potential uses, and demonstrates the impact of log format on analysis through a case study.

Findings

Log format significantly affects analysis quality

Proper log formatting enables better consumer behavior insights

Addressing log quality issues improves WAPI usability evaluation

Abstract

A Web API (WAPI) is a type of API whose interaction with its consumers is done through the Internet. While being accessed through the Internet can be challenging, mostly when WAPIs evolve, it gives providers the possibility to monitor their usage, and understand and analyze consumers' behavior. Currently, WAPI usage is mostly logged for traffic monitoring and troubleshooting. Even though they contain invaluable information regarding consumers' behavior} they are not sufficiently used by providers. In this paper, we first consider two phases of the application development lifecycle, and based on them we distinguish two different types of usage logs, namely development logs and production logs. For each of them we show the potential analyses (e.g., WAPI usability evaluation, consumers' needs identification) that can be performed, as well as the main impediments, that may be caused by the unsuitable log format. We then conduct a case study using logs of the same WAPI from different deployments and different formats, to demonstrate the occurrence of these impediments and at the same time the importance of a proper log format. Next, based on the case study results, we present the main quality issues of WAPI log data and explain their impact on data analyses. For each of them, we give some practical suggestions on how to deal with them, as well as mitigating their root cause.