Equity-Regularized Federated Multimodal Learning with Missingness-Aware Gating and Subgroup Conditional Conformal Risk Control for Clinical Prediction

Abstract

A clinical prediction model can look strong on average and still fail the patients who depend on it most. Two failure modes dominate in practice: performance that differs across patient subgroups, and quiet degradation when one data modality is routinely unavailable. Neither shows up when a model is trained for accuracy and judged by average metrics. We present PRISM-Fed (Privacy-preserving Risk Inference via Selective Multimodal fusion, Federated), an end-to-end framework in which equity is built into the federated aggregation rule and certified at the decision threshold, instead of being added as one more penalty in the loss. The framework rests on three linked mechanisms. Missingness-aware gating (MAG) conditions cross-modal attention on an explicit observability mask and a learned mask embedding; attention is renormalized over the modalities actually recorded, the pattern of absence is used as clinical information, and no imputation is needed. EqFedAvg replaces sample-size-proportional averaging with a rule that raises an institution’s aggregation weight according to the calibration deficit of its worst-served subgroup, so the global model is pulled toward the sites and populations where it performs worst. We show that this rule keeps the usual federated-averaging convergence rate, up to a bounded weight-dispersion constant. Subgroup-conditional conformal risk control (SC-CRC) fits a separate operating threshold for each protected subgroup from a federated calibration pool and gives a distribution-free, finite-sample guarantee that sensitivity is at least 1 − α in every audited subgroup. A drift-triggered loop restores this guarantee by refitting the thresholds only, while the approved model weights stay frozen. In a multi-institution evaluation covering intensive care, general ward and imaging cohorts, PRISM Fed raised worst-subgroup discrimination much more than average discrimination (worst-subgroup AUROC 0.851 versus 0.751 for federated averaging; overall AUROC 0.887 versus 0.854). It cut the expected calibration error by more than half, lowered the equalized-odds gap from 0.142 to 0.038 and reduced the share of subgroups missing the target sensitivity from 0.42 to 0.03. It also converged in about 35% fewer communication rounds and, after distillation, ran in 96 ms at 0.21 J per inference. These results suggest that fairness in clinical machine learning is better treated as a guarantee that can be audited at deployment than as a regularization problem at training time.