Measuring Equity: Funnel Representation Measurement

TL;DR

This paper introduces a methodology to measure gender representation in online product funnels using causal inference techniques, aiming to identify and understand disparities to promote equity.

Contribution

It proposes a novel funnel survival ratio metric and applies coarsened exact matching to quantify and analyze representation differences in product funnels.

Findings

The funnel survival ratio effectively detects representation disparities.

The methodology helps identify factors explaining gender differences.

It provides a framework for ongoing equity analysis in product design.

Abstract

We present a methodology to measure the gender representation for online product funnels. It is a part of the overall equity framework to better understand our products through funnel analysis. By leveraging the coarsened exact matching method from causal inference literature, we show that the funnel survival ratio metric we design can detect the representation differences inherent in our products. Understanding how big the representation differences are between different member groups, as well as understanding what explains them, is critical for fostering more equitable outcomes.