Article
Full entry | Fulltext not available
(moving wall
24 months)
Feedback
Keywords:
unconstrained optimization; diagonal quasi-Newton method; weak secant equation; global convergence
unconstrained optimization; diagonal quasi-Newton method; weak secant equation; global convergence
Summary:
We present a new diagonal quasi-Newton method for solving unconstrained optimization problems based on the weak secant equation. To control the diagonal elements, the new method uses new criteria to generate the Hessian approximation. We establish the global convergence of the proposed method with the Armijo line search. Numerical results on a collection of standard test problems demonstrate the superiority of the proposed method over several existing diagonal methods.
References:
[1] Andrei, N.: An acceleration of gradient descent algorithm with backtracking for unconstrained optimization. Numer. Algorithms 42 (2006), 63-73. DOI 10.1007/s11075-006-9023-9 | MR 2249567 | Zbl 1101.65058
[2] Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10 (2008), 147-161. MR 2424936 | Zbl 1161.90486
[3] Andrei, N.: A diagonal quasi-Newton updating method based on minimizing the measure function of Byrd and Nocedal for unconstrained optimization. Optimization 67 (2018), 1553-1568. DOI 10.1080/02331934.2018.1482298 | MR 3877965 | Zbl 1402.65049
[4] Andrei, N.: A diagonal quasi-Newton updating method for unconstrained optimization. Numer. Algorithms 81 (2019), 575-590. DOI 10.1007/s11075-018-0562-7 | MR 3953161 | Zbl 1416.49025
[5] Andrei, N.: A new accelerated diagonal quasi-Newton updating method with scaled forward finite differences directional derivative for unconstrained optimization. Optimization 70 (2021), 345-360. DOI 10.1080/02331934.2020.1712391 | MR 4207210 | Zbl 1460.90204
[6] Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16 (1966), 1-3. DOI 10.2140/pjm.1966.16.1 | MR 0191071 | Zbl 0202.46105
[7] Barzilai, J., Borwein, J. M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8 (1988), 141-148. DOI 10.1093/imanum/8.1.141 | MR 0967848 | Zbl 0638.65055
[8] Bongartz, I., Conn, A. R., Gould, N., Toint, P. L.: CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21 (1995), 123-160. DOI 10.1145/200979.201043 | Zbl 0886.65058
[9] C. G. Broyden, J. E. Dennis, Jr., J. J. Moré: On the local and superlinear convergence of quasi-Newton methods. J. Inst. Math. Appl. 12 (1973), 223-245 \99999DOI99999 10.1093/imamat/12.3.223 . DOI 10.1093/imamat/12.3.223 | MR 0341853 | Zbl 0282.65041
[10] J. E. Dennis, Jr., J. J. Moré: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comput. 28 (1974), 549-560. DOI 10.1090/S0025-5718-1974-0343581-1 | MR 0343581 | Zbl 0282.65042
[11] J. E. Dennis, Jr., J. J. Moré: Quasi-Newton methods, motivation and theory. SIAM Rev. 19 (1977), 46-89. DOI 10.1137/101900 | MR 0445812 | Zbl 0356.65041
[12] J. E. Dennis, Jr., H. Wolkowicz: Sizing and least-change secant methods. SIAM J. Numer. Anal. 30 (1993), 1291-1314. DOI 10.1137/073006 | MR 1239822 | Zbl 0802.65081
[13] Dolan, E. D., Moré, J. J.: Benchmarking optimization software with performance profiles. Math. Program. 91 (2002), 201-213. DOI 10.1007/s101070100263 | MR 1875515 | Zbl 1049.90004
[14] Farid, M., Leong, W. J., Zheng, L.: A new diagonal gradient-type method for large scale unconstrained optimization. Sci. Bull., Ser. A, Appl. Math. Phys., Politeh. Univ. Buchar. 75 (2013), 57-64. MR 3032542 | Zbl 1299.65118
[15] Gill, P. E., Murray, W.: Conjugate-gradient methods for large-scale nonlinear optimization. Technical Report SOL-79-15 Stanford University, Stanford (1979), 1-66.
[16] Goldstein, A. A.: On steepest descent. J. Soc. Ind. Appl. Math., Ser. A: Control 3 (1965), 147-151. DOI 10.1137/030301 | MR 0184777 | Zbl 0221.65094
[17] Leong, W. J., Enshaei, S., Kek, S. L.: Diagonal quasi-Newton methods via least change updating principle with weighted Frobenius norm. Numer. Algorithms 86 (2021), 1225-1241. DOI 10.1007/s11075-020-00930-9 | MR 4211118 | Zbl 1464.90099
[18] Leong, W. J., Farid, M., Hassan, M. A.: Improved Hessian approximation with modified quasi-Cauchy relation for a gradient-type method. Adv. Model. Optim. 12 (2010), 37-44. MR 2591783 | Zbl 1332.90346
[19] Nash, S. G.: Preconditioning of truncated-Newton methods. SIAM J. Sci. Stat. Comput. 6 (1985), 599-616. DOI 10.1137/0906042 | MR 0791188 | Zbl 0592.65038
[20] Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35 (1980), 773-782. DOI 10.1090/S0025-5718-1980-0572855-7 | MR 0572855 | Zbl 0464.65037
[21] Nocedal, J., Wright, S. J.: Numerical Optimization. Springer Series in Operations Research and Financial Engineering. Springer, New York (2006). DOI 10.1007/b98874 | MR 2244940 | Zbl 1104.65059
[22] Powell, M. J. D.: A new algorithm for unconstrained optimization. Nonlinear Programming Elsevier, Amsterdam (1970), 31-65 \99999DOI99999 10.1016/B978-0-12-597050-1.50006-3 . MR 0272162 | Zbl 0228.90043
[23] Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7 (1997), 26-33. DOI 10.1137/S1052623494266365 | MR 1430555 | Zbl 0898.90119
[24] Wolfe, P.: Convergence conditions for ascent methods. SIAM Rev. 11 (1969), 226-235. DOI 10.1137/101103 | MR 0250453 | Zbl 0177.20603
[25] Zhu, M., Nazareth, J. L., Wolkowicz, H.: The quasi-Cauchy relation and diagonal updating. SIAM J. Optim. 9 (1999), 1192-1204. DOI 10.1137/S1052623498331793 | MR 1724783 | Zbl 1013.90137
Search
Partner of
