Multi-Objective Optimization of Predictive Learning Models Under Uncertain Data Distributions

Abstract

Adaptive learning approaches are becoming more common in situations where they face a distribution shift, which means that the distributions at deployment time may be different from the reference distribution. Fixed objective weights, fixed ambiguity radius, and fixed learning rates can all be ill-suited to handle the case if such shifts keep happening. This work offers an Adaptive Lyapunov-Controlled Multi-Objective Min-Max Learning algorithm that views robust predictive learning as a closed-loop dynamical system and controls objective weights, ambiguity radius, and learning rates based on three observable states: distribution shift, predictive volatility, and parameter change. Comparison is done between the new approach and empirical risk minimization, traditional DRO, fixed weight multi-objective optimization, adaptive weighting without Lyapunov control, and adaptive uncertainty radius control. With the new methodology, we are able to get the following figures: nominal loss = 0.168; worst case loss = 0.291; loss volatility = 0.021; parameter motion = 0.014; OOD degradation = +18.3%. In the ablation study, when we remove the Lyapunov damping, the local spectral radius will be 0.82 versus 1.14, and loss volatility will be 0.021 versus 0.098. From this observation, it is clear that joint feedback is capable of controlling the trade offs in robustness vs accuracy vs stability within our simulation environment.