Advancing ICU decision making through multimodal data fusion in predicting cardiovascular disorders

Krones F., Papastylianou T., Qian G., Parsons G., Shukla B., Papież BW., Mahdi A.

Although ICUs generate rich multimodal data, most studies examine only narrow modality combinations, limiting progress toward clinically useful decision-support systems. We propose a systematic framework for evaluating multimodal fusion strategies in the ICU, enabling structured comparisons of early, intermediate and late fusion across combinations of imaging, tabular, text and time-series data. To assess generalisability across diagnoses and patient groups, we apply the framework to six cardiovascular conditions covering a broad diagnostic spectrum and report performance broken down by age, sex and insurance. Rather than asking whether multimodal fusion helps, we ask under which conditions it helps. We find that multimodal models frequently outperform unimodal baselines, with gains of up to five percentage points in AUC, particularly when modalities provide complementary information or when prediction uncertainty is high. Across the architectures we explored, late fusion is the most robust and best-performing strategy. Using uncertainty-based ranking, we show that multimodal integration is especially effective for correcting high-uncertainty cases, substantially reducing both false-negative and false-positive rates relative to single-modality models. The most informative data are typically available within the first 24 hours of ICU admission. These findings offer guidance for prioritising modalities and deploying multimodal models in real-world clinical settings.

DOI

10.1016/j.inffus.2026.104573

Type

Journal article

Publication Date

2026-12-01T00:00:00+00:00

Volume

136

Permalink More information Close