Dynamic Information Sub-Selection for Decision Support
Hung-Tien Huang, Maxwell Lennon, Shreyas Bhat Brahmavar, Sean Sylvia,, Junier B. Oliva
TL;DR
This paper presents DISS, a framework that dynamically selects relevant information to improve black-box decision-making, demonstrating superior performance across multiple AI-assisted decision tasks.
Contribution
DISS introduces a scalable, data-efficient approach for tailoring information to black-box decision-makers, enhancing their performance and interpretability.
Findings
Outperforms state-of-the-art methods in various applications
Improves decision accuracy with dynamic feature selection
Enhances interpretability and resource efficiency
Abstract
We introduce Dynamic Information Sub-Selection (DISS), a novel framework of AI assistance designed to enhance the performance of black-box decision-makers by tailoring their information processing on a per-instance basis. Blackbox decision-makers (e.g., humans or real-time systems) often face challenges in processing all possible information at hand (e.g., due to cognitive biases or resource constraints), which can degrade decision efficacy. DISS addresses these challenges through policies that dynamically select the most effective features and options to forward to the black-box decision-maker for prediction. We develop a scalable frequentist data acquisition strategy and a decision-maker mimicking technique for enhanced budget efficiency. We explore several impactful applications of DISS, including biased decision-maker support, expert assignment optimization, large language model…
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
- There is potential for broader applicability of this general purpose approach. - The authors begin to demonstrate generality by the diversity of experimental setups in this work.
- The Mimic approach assumes the reward function r is known, yet the paper does not justify why this is realistic, especially for human-in-the-loop scenarios. - The paper tries to address too many applications (e.g., interpretability, human decision-making, and LLM decision support) without properly justifying and motivating each one. - The experiments rely primarily on standard UCI tabular datasets, where a well-trained model could already achieve high performance, the evaluation set up feels a
The authors have a strongly motivated piece. I found the paper generally well-written and appreciated the rich and thoughtful connections to people's cognitive constraints. I thought the authors took on quite an ambitious, wide-ranging suite of decision making settings to evaluate across (e.g., I very much enjoyed that the authors directly connected back --- and concretely empirically connected back --- to the points they raised in their "Applications" section on page 2 [something not done in en
Perhaps the biggest weakness of the current work is some confusion in the set-up of DISS and Mimic. I found myself needing to reread the text quite a few times to delineate what DISS is relative to Mimic (and what the authors' contributions are therein?) And while the empirical work is strong, the Modiste baseline (as the authors note) was not really designed to extend to the kinds of tasks the authors considered here wrt the number of possible actions (hence is quite a weak/ill-suited baselin
* **[S1] Novel and Well-Motivated Methodological Core:** The mimic-structured regression elegantly factorizes the learning problem, backed by clear theoretical intuition and proof. * **[S2] Impressive Generality and Unification:** A key strength is the successful unification of four distinct and important application areas under the single DISS framework. This demonstrates the flexibility and broad relevance of the proposed approach, framing it not as a niche solution but as a general-purpose r
* **[W1] Gap Between Theory and Practice (Adaptivity):** The main theoretical weakness is the mismatch between the assumptions of the risk bound analysis and the practical implementation. The proof in Appendix A assumes that the observations used to train the mimic model $\hat{M}$ are i.i.d. However, the FTS algorithm collects these observations *adaptively*, which breaks the i.i.d. assumption. While the empirical results are strong, bridging this theoretical gap (perhaps by discussing how the a
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Fuzzy Logic and Control Systems · Data Mining Algorithms and Applications