Robust Optimization of Learning Models Under Distributional and Parametric Uncertainty

Abstract

Learning models in uncertain environments are subjected to many uncertainties, such as uncertainty in the data distribution and uncertainty in the model parameters. These uncertainty sources are usually handled separately in conventional robust learning methods, or with fixed robustness levels, causing robustness to be allocated ineffectively and introducing conservatism. In this work, a sensitivity-adaptive robust learning approach that jointly accounts for both distributional and parametric uncertainty with a shared robustness budget is proposed. Distributional uncertainty is described by a Wasserstein ambiguity set, whereas parametric uncertainty is described by an ellipsoidal uncertainty set. The proposed framework quantifies the relative sensitivity of the learning objective with respect to these uncertainty sources and allocates the robustness budget in response to the changing uncertainty sensitivity profile. A mathematical formulation is provided to explain the integrated robust-learning objective, sensitivity-based allocation mechanism, joint worst-case risk and projected adaptive update. Compared with empirical risk minimization, Wasserstein distributionally robust optimization, parametric robust learning and fixed coupled robustness, a numerical analysis is carried out. Results show that the sensitivity-adaptive allocation can minimize robust risk while providing a more balanced treatment of distributional uncertainty and parametric uncertainty. The allocation is also steadily convergent, with the sensitivity profile remaining steady. The study offers a coherent framework for handling several uncertainty sources in learning models and thus a basis for adaptive robustness in uncertain learning contexts.