Robust Tests in Online Decision-Making
Gi-Soo Kim, Hyun-Joon Yang, Jane P. Kim

TL;DR
This paper introduces a robust actor-critic bandit algorithm and a new testing procedure to evaluate actor parameters, addressing critic misspecification issues in sequential decision-making, especially relevant for mobile health applications.
Contribution
It proposes a modified actor-critic algorithm that is robust to critic misspecification and develops a novel testing procedure for actor parameters under such conditions.
Findings
The new algorithm maintains performance despite critic misspecification.
The testing procedure provides valid inference even with model misspecification.
Application to mobile health demonstrates practical utility.
Abstract
Bandit algorithms are widely used in sequential decision problems to maximize the cumulative reward. One potential application is mobile health, where the goal is to promote the user's health through personalized interventions based on user specific information acquired through wearable devices. Important considerations include the type of, and frequency with which data is collected (e.g. GPS, or continuous monitoring), as such factors can severely impact app performance and users' adherence. In order to balance the need to collect data that is useful with the constraint of impacting app performance, one needs to be able to assess the usefulness of variables. Bandit feedback data are sequentially correlated, so traditional testing procedures developed for independent data cannot apply. Recently, a statistical testing procedure was developed for the actor-critic bandit algorithm. An…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Data Stream Mining Techniques · Mobile Crowdsensing and Crowdsourcing
MethodsTest · Greedy Policy Search
