ADWIN-Guided Simulated Annealing–Optimized Online Gradient Boosting for Malware Detection under Controlled Concept Drift
Keywords:
ADWIN, concept drift, gradient boosting,LightGBM, malware detection, online learning, Portable Executable (PE) malware, simulated annealing, streaming classification, EMBER 2018.Abstract
Machine-learning-based malware detectors can suffer performance degradation when malicious software evolves and the relationship between Portable Executable (PE) features and class labels changes. This work proposes an ADWIN-Guided Simulated Annealing–Optimized Online Gradient Boosting framework for malware detection under controlled concept drift. The framework employs LightGBM as the gradient-boosting classifier, ADWIN to monitor the online prediction-error stream and identify statistically significant drift, and simulated annealing to optimize model hyperparameters and the malware decision threshold after a drift alarm. Experiments were conducted using a balanced 120,000-instance subset of EMBER 2018 version-2 static PE features, comprising 2,381 numeric features. The data were divided into 50,000 initial-learning instances, 50,000 controlled streaming instances, and 20,000 independent holdout instances. Gradual feature drift was reproducibly injected into 25% of malware instances in the latter half of the stream. ADWIN detected the first drift event at stream index 25,983, corresponding to a detection delay of 983 samples after the injected onset. Following drift-triggered simulated annealing optimization and model adaptation, the proposed approach achieved 96.12% post-alarm F1-score, 97.82% malware recall, and 99.44% AUROC, compared with 94.70%, 93.97%, and 98.91%, respectively, for the static LightGBM baseline. The adaptive model reduced mean false negatives from 30.24 to 10.96 per 1,000-sample batch, representing a 63.76% reduction in missed malware, while the false-positive rate increased from 4.44% to 5.75%. On the independent clean holdout set, the adapted model retained 95.67% F1-score and 99.32% AUROC. The findings demonstrate that ADWIN-triggered simulated-annealing adaptation can improve malware-detection resilience under controlled gradual feature drift.
Downloads
Downloads
Published
Issue
Section
License
You are free to:
- Share — copy and redistribute the material in any medium or format for any purpose, even commercially.
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
- The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
- Attribution — You must give appropriate credit , provide a link to the license, and indicate if changes were made . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Notices:
You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation .
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.

