More Reviews May Not Help: Evidence from Incentivized First Reviews on Airbnb

TL;DR

This study uses a large-scale randomized experiment on Airbnb to show that incentivized reviews increase quantity but tend to be more negative, with no clear impact on overall revenue or quality of transactions.

Contribution

It provides empirical evidence on how incentivized reviews affect review quantity, sentiment, and transaction quality in online marketplaces.

Findings

Incentivized reviews increase review quantity.

Reviews tend to be more negative under incentives.

No significant effect on total revenue or nights sold.

Abstract

Online reviews are typically written by volunteers and, as a consequence, information about seller quality may be under-provided in digital marketplaces. We study the extent of this under-provision in a large-scale randomized experiment conducted by Airbnb. In this experiment, buyers are offered a coupon to review listings that have no prior reviews. The treatment induces additional reviews and these reviews tend to be more negative than reviews in the control group, consistent with selection bias in reviewing. Reviews induced by the treatment result in a temporary increase in transactions but these transactions are for fewer nights, on average. The effects on transactions and nights per transaction cancel out so that there is no detectable effect on total nights sold and revenue. Measures of transaction quality in the treatment group fall, suggesting that incentivized reviews do not improve matching. We show how market conditions and the design of the reputation system can explain our findings.