Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Data
Yang Sui, Qi Xu, Yang Bai, and Annie Qu
TL;DR
This paper introduces a novel two-step multi-task learning approach that effectively handles block-wise missing data and heterogeneity across multiple sources, improving predictive performance in complex datasets.
Contribution
The paper presents a unified framework combining block-wise data imputation and disentangled shared and task-specific mappings for heterogeneous multi-source MTL.
Findings
Outperforms existing methods in numerical experiments
Demonstrates effectiveness on ADNI real-data analysis
Addresses multiple heterogeneity types simultaneously
Abstract
Multi-task learning (MTL) has emerged as an imperative machine learning tool to solve multiple learning tasks simultaneously and has been successfully applied to healthcare, marketing, and biomedical fields. However, in order to borrow information across different tasks effectively, it is essential to utilize both homogeneous and heterogeneous information. Among the extensive literature on MTL, various forms of heterogeneity are presented in MTL problems, such as block-wise, distribution, and posterior heterogeneity. Existing methods, however, struggle to tackle these forms of heterogeneity simultaneously in a unified framework. In this paper, we propose a two-step learning strategy for MTL which addresses the aforementioned heterogeneity. First, we impute the missing blocks using shared representations extracted from homogeneous source across different tasks. Next, we disentangle the…
Peer Reviews
Decision·Submitted to ICLR 2025
The paper addresses a well-motivated and pervasive problem in large-scale data analysis, namely the integration of block-wise missing data from distinct sources. The proposed imputation method is a novel application of the encoder-decoder framework that, to my knowledge, is new to the missing data literature. Numerical results indicate that this may be a promising approach for imputation.
The primary weakness of this work is in the evaluation of the proposed MTL-HMB method. The proposed simulation setting (described in Appendix A.2) is far too small and simple to necessitate the heavy machinery used by the proposed method and the STL and HTL methods also applied to the data. 1. The samples sizes are too small relative to the trained neural networks to draw meaningful conclusions from the simulations. This is most evident in Figure 5d: as n grows, the performance of the single-t
This paper is well-structured and coherent.
1.Why does Equation 1 minimize only L_pre? 2.The authors utilize a bar chart to display qualitative results, however, quantitative results would be more appropriate, enabling the reader to make numerical comparisons. 3.The experiments conducted by the authors were insufficient, resulting in an incomplete evaluation of the model's performance.
The authors conducted comprehensive numerical experiments, validating the efficacy of their method across various levels of heterogeneity. This adds robustness to their claims and provides confidence in the generalizability of the results. The two-step approach is clearly defined, and the methods used for imputing missing data and disentangling mappings are appropriately justified.
1.The authors do not provide a detailed explanation of how the proposed method specifically addresses block-wise datasets in the paper. 2.The authors claim in the paper that they use a shared feature extraction encoder and a task-specific feature extraction encoder. What are the differences between these two, and how are they reflected in the methodology? 3.Why are some formulas numbered while others are not? The authors need to revise and check this. 4.The authors propose several loss functi
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCOVID-19 diagnosis using AI · Domain Adaptation and Few-Shot Learning · Anomaly Detection Techniques and Applications