Error-bound analysis and convergence characterization of numerical optimization algorithms in engineering systems

Abstract

Numerical optimization methods are widely applied in engineering systems in solving complicated engineering problems related to nonlinear modeling, structural design, parameter estimation, and constrained optimization. However, attaining highly accurate solutions with assured convergence is difficult because of issues with error propagation, computational complexity, and sensitivity to system parameters. This study proposes an analysis approach for estimating the error bounds and convergence properties of numerical optimization methods in engineering applications. Mathematical conditions such as smoothness of objective function, strong convexity, Polyak-Łojasiewicz inequality, Lipschitz property of Hessian, and composite optimization formulations are used in formulating convergence results and error properties. Sufficient decrease conditions, linear convergence bounds of gradient methods, explicit error-distance relations, local quadratic convergence of Newton methods, and convergence of proximal-gradient optimization are formulated theoretically. The numerical validation process relies on the analysis of structural compliance optimization and nonlinear stiffness-calibration problems. From the gradient-based optimization analysis, geometric error minimization is observed from a function value difference that ranges from 5.175 × 10⁻² to 1.694 × 10⁻¹⁴ in 40 iterations; thus, illustrating the reliability of the convergence bound derivation. Error-bound analysis reveals that there is a close relation between theoretical distance-to-optimum and design variable error; hence, showing a decreasing bound ratio during iterative optimization. With respect to the non-linear calibration problem, Newton optimization minimizes the solution error from 5.831 × 10⁻¹ to 6.554 × 10⁻⁵, thereby revealing quadratic convergence after the local convergence region. It is also clear from the convergence analysis that gradient descent has O (κ log(1/ε)) complexity while Newton optimization gives O (log log(1/ε)) local convergence efficiency.