Knowledge Gradient for Selection with Covariates: Consistency and Computation

TL;DR

This paper extends the knowledge gradient method to ranking and selection problems with covariates, proving its consistency and proposing a stochastic gradient algorithm for efficient computation.

Contribution

It introduces a knowledge gradient-based sampling policy for covariate-dependent selection, proving its almost sure consistency and providing a practical computation method.

Findings

The policy is consistent under minimal assumptions.

The stochastic gradient algorithm effectively computes the policy.

Numerical experiments demonstrate the method's performance.

Abstract

Knowledge gradient is a design principle for developing Bayesian sequential sampling policies to solve optimization problems. In this paper we consider the ranking and selection problem in the presence of covariates, where the best alternative is not universal but depends on the covariates. In this context, we prove that under minimal assumptions, the sampling policy based on knowledge gradient is consistent, in the sense that following the policy the best alternative as a function of the covariates will be identified almost surely as the number of samples grows. We also propose a stochastic gradient ascent algorithm for computing the sampling policy and demonstrate its performance via numerical experiments.