FraudFox: Adaptable Fraud Detection in the Real World

TL;DR

FraudFox is an adaptable, scalable fraud detection system that dynamically combines risk assessments, optimizes decision-making under constraints, and adapts to evolving fraudster behavior, proven effective in Amazon's production environment.

Contribution

This work introduces FraudFox, a novel system that integrates dynamic oracle-weighting, optimal decision surfaces, and Pareto analysis for real-world fraud detection and resource management.

Findings

Implemented in Amazon, improves fraud detection performance

Effectively adapts to changing fraudster strategies

Enhances resource allocation for investigations

Abstract

The proposed method (FraudFox) provides solutions to adversarial attacks in a resource constrained environment. We focus on questions like the following: How suspicious is `Smith', trying to buy \$500 shoes, on Monday 3am? How to merge the risk scores, from a handful of risk-assessment modules (`oracles') in an adversarial environment? More importantly, given historical data (orders, prices, and what-happened afterwards), and business goals/restrictions, which transactions, like the `Smith' transaction above, which ones should we `pass', versus send to human investigators? The business restrictions could be: `at most $x$ investigations are feasible', or `at most \$$y$ lost due to fraud'. These are the two research problems we focus on, in this work. One approach to address the first problem (`oracle-weighting'), is by using Extended Kalman Filters with dynamic importance weights, to automatically and continuously update our weights for each 'oracle'. For the second problem, we show how to derive an optimal decision surface, and how to compute the Pareto optimal set, to allow what-if questions. An important consideration is adaptation: Fraudsters will change their behavior, according to our past decisions; thus, we need to adapt accordingly. The resulting system, \method, is scalable, adaptable to changing fraudster behavior, effective, and already in \textbf{production} at Amazon. FraudFox augments a fraud prevention sub-system and has led to significant performance gains.