Predictive Coding beyond Correlations
Tommaso Salvatori, Luca Pinchetti, Amine M'Charrak, Beren Millidge,, Thomas Lukasiewicz
TL;DR
This paper demonstrates that predictive coding algorithms can perform causal inference tasks, including interventions and causal graph inference, and improve image classification by leveraging causal reasoning.
Contribution
It introduces modifications to predictive coding enabling causal inference without altering causal graphs and applies it to unknown graph scenarios and image classification.
Findings
Predictive coding can perform interventions without graph mutilation.
Models can infer causal graphs from observational data.
Enhanced predictive coding improves image classification performance.
Abstract
Recently, there has been extensive research on the capabilities of biologically plausible algorithms. In this work, we show how one of such algorithms, called predictive coding, is able to perform causal inference tasks. First, we show how a simple change in the inference process of predictive coding enables to compute interventions without the need to mutilate or redefine a causal graph. Then, we explore applications in cases where the graph is unknown, and has to be inferred from observational data. Empirically, we show how such findings can be used to improve the performance of predictive coding in image classification tasks, and conclude that such models are able to perform simple end-to-end causal inference tasks.
Peer Reviews
Decision·ICML 2024 Poster
- Original exploration of PC concepts in this context
- Paper is very informally written and theorems / definitions are not up to standards of a statistics community. Eg Theorem 1 has no assumptions stated, what are the spaces allowed of the variables involved etc. - **I don't see actual methodological novelty wrt to causal inference and/or discovery except "interpretations" of existing techniques in the parlance of predictive coding.** If the interpretation is the only contribution, can you clarify what is the value here? - Theorem 1 is irreleva
- The paper provides another way of answering causal queries from the conventional methods, which is interesting in itself
- The main weakness of the work is the presentation, in that it is very hard to parse the information. A lot of the details are in the Appendix but there are not enough pointers to the Appendix in the main text (e.g. Appendix D is never referenced). - Where are the results comparing the causal queries with VACA, CAREFL, etc? It's not stated in the main text. - Results in section 3.2. For which experiment are these results of? It seems its only for counterfactual queries. - Figures like fig 4
- The paper is self-contained with a pretty novel approach for causal inference. - The proposed method outperforms existing methods for structural learning, interventional, and counterfactual inferencing. - The authors claim that the proposed method is parameter efficient and does not require extensive hyperparameter tuning.
- It is not very obvious how the proposed method compares with the existing frameworks. So, it is also not very clear how the proposed method would contribute to the research direction. An in-depth discussion comparing the PC graph with DAG might lend more weight to the paper's influence on the causality. - It is not easy to see in principle how the proposed method is superior to the probabilistic method. - While the comparison against baseline methods using synthetic data provides some insights
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBayesian Modeling and Causal Inference · Neural dynamics and brain function · EEG and Brain-Computer Interfaces