Minimax-Based robust optimization of cognitive engagement prediction in adaptive online learning systems

Abstract

Adaptive online learning systems use predictions of learners' cognitive engagement to determine when to intervene, but engagement predictors are trained by average loss minimization for the majority group of learners and used on changing populations with interaction logs corrupted by device-specific noise. formalize the engagement prediction task as a hierarchical minimax problem where the learner model plays against two adversary players who alter the proportions of device-discipline groups around the observed mixture within Kullback-Leibler neighborhood and add per-learner feature-specific noise based on the test-retest variability estimate. For the case of device-aware linear scorers, both maxima in the minimax game have closed forms: the first one corresponds to a loss-weighted l1 margin penalty and the second one is the log-sum-exp of cohort losses. Thus, the saddle-point problem can be solved as a convex program that is trained as empirically risk minimization and has equilibrium by Sion's theorem. On a 16-feature learner-interaction benchmark using six cohorts, with a mobile dominant shifted distribution and attacks in the worst-case telemetry setting, and validation-based model selection for each method across ten random seeds, the proposed Hierarchical Minimax Robust predictor (HMR) increases shifted robust accuracy by 0.655 - 0.643 = 0.012 over empirical risk minimisation in all ten seeds (p=0.002) and in-distribution robust worst-group accuracy by 0.648 - 0.627 = 0.021 (p=0.004), and beats importance weighting, Group DRO and CVaR in robust accuracy by 1.05 to 1.61 points, at a cost of only 0.38 points of in-distribution accuracy and without any impact on shifted accuracy or calibration. HMR achieves performance similar to adversarial training but uses a smaller perturbation budget. Ablations demonstrate that the telemetry adversary provides the robustness benefit, the cohort adversary with regularisation maintains accuracy under distribution shift, and noise-aware bounds converge to the uniform ball with homogeneous logging noise.