Learning Predictive Checklists with Probabilistic Logic Programming
Yukti Makhija, Edward De Brouwer, Rahul G. Krishnan

TL;DR
This paper introduces a novel probabilistic logic programming approach for automatically learning predictive checklists from diverse data types, enhancing interpretability and performance in complex prediction tasks.
Contribution
It presents a new method that learns predictive checklists from various data modalities using probabilistic logic programming, allowing for flexible interpretability and improved accuracy.
Findings
Outperforms existing explainable ML techniques on image sequence data
Effective in handling diverse data types like time series and clinical notes
Provides a tunable tradeoff between interpretability and data fidelity
Abstract
Checklists have been widely recognized as effective tools for completing complex tasks in a systematic manner. Although originally intended for use in procedural tasks, their interpretability and ease of use have led to their adoption for predictive tasks as well, including in clinical settings. However, designing checklists can be challenging, often requiring expert knowledge and manual rule design based on available data. Recent work has attempted to address this issue by using machine learning to automatically generate predictive checklists from data, although these approaches have been limited to Boolean data. We propose a novel method for learning predictive checklists from diverse data modalities, such as images and time series. Our approach relies on probabilistic logic programming, a learning paradigm that enables matching the discrete nature of checklist with…
Peer Reviews
Decision·Submitted to ICLR 2024
- Originality: This work extends checklist learning to data modalities other than tabular data and combines the power of deep learning with the interpretability of checklist models. The proposed method is interesting. - Clarity & Quality: The background and methodology are clearly presented. The paper is easy to follow. - Significance: The proposed method seems to be a practical solution to the problem of learning checklist models from raw data modalities. Such models, if learned successfully, m
- There are too many hyperparameters in the proposed method, including the weight of the regularization terms, the number of concepts, and the threshold $T$, in addition to the architecture details of the neural networks. The authors should provide some guidance on how to choose these hyperparameters. - The learned "concept"s are hard to interpret from my point of view. The authors suggest that the concepts can be sensed by using post hoc attribution methods. However, it is well-known that the a
- Originality: The paper demonstrates originality in creative combinations of existing ideas and approaches to the target problem - Clarity: Problem formulations and related works are clearly described and cited. The paper is well-organized with most components including limitations. - Significance: Classification performances are reported with multiple metrics and confidence intervals
1. One weakness is the results discussion using MNIST data only, which is not so intuitive in the checklist concept motivated by healthcare examples in the introduction part. And the paper has results from clinical data of PhysioNet and MIMIC III in the supplementary materials, which should be much better than the MNIST story. The necessity of using checklist, instead of other benchmark methods, on the experiment data tasks (especially non-healthcare MNIST data) is another question not explained
-The « related works » analysis is thorough and seems up-to-date. -I found the experiment subsection 5.1 truly compelling. Many metrics are reported, which I think is not done often enough. -The approach is well-explained and flexible.
**Major** 1 – Most how the points I would like to raise concern interpretability. (See also the points in the Question section on that matter.) 1.1 - Interpretability is directly impacted by the complexity of the model itself, but the fact that the algorithm is in itself a black box makes it such that understanding why the model is what it is is unreachable. 1.2 – As discussed in [1], p.17, when it comes to logical rules, the more digits there are to take into account, the less the rule is in
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLogic, Reasoning, and Knowledge · Semantic Web and Ontologies · Rough Sets and Fuzzy Logic
