Second-Order variational algorithm design for accelerated numerical optimization in engineering applications

Abstract

Engineering optimization applications usually consist of a nonlinear objective function, an ill-conditioned optimization landscape, physical constraints, and uncertain operating conditions; therefore, efficient numerical optimization is not easy to achieve. First-order methods use gradient-based techniques, while second-order methods can be expensive due to the calculation of the Hessian matrix and its factorization. The Second Order Variational Minimax Algorithm (SOVMA) proposed in this paper combines variational optimization, curvature-based search directions, adaptive step-length control, constraint handling, and minimax uncertainty handling within a single framework. The algorithm uses the information of the regularized Hessian matrix to calculate the second-order search directions, while the unfavorable operating conditions of the problem are calculated by means of an inner maximization algorithm. Optimization performed with such an algorithm tries to find a feasible point that minimizes the worst-case engineering objective function. Numerical simulations are performed on Rosenbrock, Rastrigin, Ackley, Himmelblau, and ill-conditioned quadratic benchmark test problems, followed by a case study on uncertain thermal energy systems design. The comparative results presented indicate that the SOVMA method needs only 31 iterations, whereas Newton needs 42, Trust-Region needs 49, BFGS needs 74, Conjugate Gradient needs 151, and Gradient Descent needs 286 iterations. Furthermore, the SOVMA method attains the best average and worst-case objective values of 1.181 and 1.542, respectively. The ablation study proves the significance of second-order information, step adaptivity, and minimax robustness. The outcomes prove that SOVMA offers faster convergence and enhanced decision robustness for engineering applications.