Article
| Title: | An accelerated gradient descent method based on a non-monotone backtracking line search scheme for unconstrained optimization and image restoration problems (English) |
| Author: | Ashrafi, Ali |
| Author: | Mirzaei, Seyed Hamzeh |
| Language: | English |
| Journal: | Applications of Mathematics |
| ISSN: | 0862-7940 (print) |
| ISSN: | 1572-9109 (online) |
| Volume: | 70 |
| Issue: | 5 |
| Year: | 2025 |
| Pages: | 711-728 |
| Summary lang: | English |
| . | |
| Category: | math |
| . | |
| Summary: | This study introduces an accelerated gradient descent method based on a non-monotone backtracking line search scheme. A simple adaptive quadratic model is enhanced by utilizing a real, positive definite scalar matrix derived from the Taylor expansion of the objective function, rather than relying on the exact Hessian. The global and superlinear convergence of the defined model is established under appropriate conditions. Numerical experiments on a set of standard unconstrained optimization problems and image restoration problems show that the new algorithm outperforms other comparable methods in terms of efficiency and robustness. (English) |
| Keyword: | unconstrained optimization |
| Keyword: | gradient descent method |
| Keyword: | line search |
| Keyword: | simple quadratic model |
| MSC: | 49M37 |
| MSC: | 90C30 |
| MSC: | 90C53 |
| DOI: | 10.21136/AM.2025.0131-25 |
| . | |
| Date available: | 2025-11-07T18:05:22Z |
| Last updated: | 2025-11-16 |
| Stable URL: | http://hdl.handle.net/10338.dmlcz/153156 |
| . | |
| Reference: | [1] Ahookhosh, M., Amini, K.: An efficient nonmonotone trust-region method for unconstrained optimization.Numer. Algorithms 59 (2012), 523-540. Zbl 1243.65066, MR 2892563, 10.1007/s11075-011-9502-5 |
| Reference: | [2] Andrei, N.: An acceleration of gradient descent algorithm with backtracing for unconstrained optimization.Numer. Algorithms 42 (2006), 63-73. Zbl 1101.65058, MR 2249567, 10.1007/s11075-006-9023-9 |
| Reference: | [3] Andrei, N.: An unconstrained optimization test functions collection.Adv. Model. Optim. 10 (2008), 147-161. Zbl 1161.90486, MR 2424936 |
| Reference: | [4] Andrei, N.: Open problems in nonlinear conjugate gradient algorithms for unconstrained optimization.Bull. Malays. Math. Sci. Soc. (2) 34 (2011), 319-330. Zbl 1225.49030, MR 2788405 |
| Reference: | [5] Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives.Pac. J. Math. 16 (1966), 1-3. Zbl 0202.46105, MR 0191071, 10.2140/pjm.1966.16.1 |
| Reference: | [6] Conn, A. R., Gould, N. I. M., Toint, P. L.: Trust-Region Methods.MPS/SIAM Series on Optimization 1. SIAM, Philadelphia (2000). Zbl 0958.65071, MR 1774899, 10.1137/1.9780898719857 |
| Reference: | [7] Dolan, E. D., Moré, J. J.: Benchmarking optimization software with performance profiles.Math. Program. 91 (2002), 201-213. Zbl 1049.90004, MR 1875515, 10.1007/s101070100263 |
| Reference: | [8] Goldstein, A. A.: On steepest descent.J. Soc. Ind. Appl. Math., Ser. A, Control 3 (1965), 147-151. Zbl 0221.65094, MR 0184777, 10.1137/0303013 |
| Reference: | [9] Gonzales, R. C., Woods, R. E.: Digital Image Processing.Prentice Hall, Upper Saddle River (2008). |
| Reference: | [10] Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton's method.SIAM. J. Numer. Anal. 23 (1986), 707-716. Zbl 0616.65067, MR 0849278, 10.1137/0723046 |
| Reference: | [11] Hager, W., Zhang, H.: A survey of nonlinear conjugate gradient methods.Pac. J. Optim. 2 (2006), 35-58. Zbl 1117.90048, MR 2548208 |
| Reference: | [12] Hestenes, M. R., Stiefel, E.: Methods of conjugate gradients for solving linear systems.J. Res. Natl. Bur. Stand. 49 (1952), 409-436. Zbl 0048.09901, MR 0050378, 10.6028/jres.049.044 |
| Reference: | [13] Ivanov, B., Stanimirović, P. S., Milovanović, G. V., Djordjević, S., Brajević, I.: Accelerated multiple step-size methods for solving unconstrained optimization problems.Optim. Methods Softw. 36 (2021), 998-1029. Zbl 1489.65085, MR 4432929, 10.1080/10556788.2019.1653868 |
| Reference: | [14] Li, D.-H., Fukushima, M., Qi, L., Yamashita, N.: Regularized Newton methods for convex minimization problems with singular solutions.Comput. Optim. Appl. 28 (2004), 131-147. Zbl 1056.90111, MR 2072529, 10.1023/B:COAP.0000026881.96694.32 |
| Reference: | [15] Li, Q., Zheng, B., Zheng, Y.: An efficient nonmonotone adaptive cubic regularization method with line search for unconstrained optimization problem.Appl. Math. Lett. 98 (2019), 74-80. Zbl 1423.90141, MR 3959825, 10.1016/j.aml.2019.05.040 |
| Reference: | [16] Mirzaei, S. H., Ashrafi, A.: Correction of nonmonotone trust region algorithm based on a modified diagonal regularized quasi-Newton method.J. Inequal. Appl. 2024 (2024), Article ID 90, 22 pages. Zbl 1546.49058, MR 4768662, 10.1186/s13660-024-03161-x |
| Reference: | [17] Niri, T. D., Amini, K.: Effective nonmonotone trust region method based on a simple cubic model for unconstrained optimization problems.J. Appl. Math. Comput. 71 (2025), 707-723. Zbl 08078311, MR 4854953, 10.1007/s12190-024-02241-x |
| Reference: | [18] Nocedal, J., Wright, S. J.: Numerical Optimization.Springer Series in Operations Research and Financial Engineering. Springer, New York (2006). Zbl 1104.65059, MR 2244940, 10.1007/978-0-387-40065-5 |
| Reference: | [19] Petrović, M. J.: An accelerated double step size model in unconstrained optimization.Appl. Math. Comput. 250 (2015), 309-319. Zbl 1328.90115, MR 3285540, 10.1016/j.amc.2014.10.104 |
| Reference: | [20] Petrović, M. J., Stanimirović, P. S.: Accelerated double direction method for solving unconstrained optimization problems.Math. Probl. Eng. 2014 (2014), Article ID 965104, 8 pages. Zbl 1407.90262, MR 3193716, 10.1155/2014/965104 |
| Reference: | [21] Polyak, R. A.: Regularized Newton method for unconstrained convex optimization.Math. Program. 120 (2009), 125-145. Zbl 1189.90121, MR 2496429, 10.1007/s10107-007-0143-3 |
| Reference: | [22] Stanimirović, P. S., Ivanov, B., Ma, H., Mosić, D.: A survey of gradient methods for solving nonlinear optimization.Electron. Res. Arch. 28 (2020), 1573-1624. Zbl 1458.65067, MR 4182341, 10.3934/era.2020115 |
| Reference: | [23] Stanimirović, P. S., Miladinović, M. B.: Accelerated gradient descent methods with line search.Numer. Algorithms 54 (2010), 503-520. Zbl 1198.65104, MR 2670666, 10.1007/s11075-009-9350-8 |
| Reference: | [24] Toint, P. L.: An assessment of nonmonotone line search techniques for unconstrained optimization.SIAM. J. Sci. Comput. 17 (1996), 725-739. Zbl 0849.90113, MR 1384262, 10.1137/S106482759427021X |
| Reference: | [25] Ueda, K., Yamashita, N.: A regularized Newton method without line search for unconstrained optimization.Comput. Optim. Appl. 59 (2014), 321-351. Zbl 1302.90218, MR 3253745, 10.1007/s10589-014-9656-x |
| Reference: | [26] Wolfe, P.: Convergence conditions for ascent methods.SIAM Rev. 11 (1969), 226-235. Zbl 0177.20603, MR 0250453, 10.1137/1011036 |
| Reference: | [27] Yu, G., Huang, J., Zhou, Y.: A descent spectral conjugate gradient method for impulse noise removal.Appl. Math. Lett. 23 (2010), 555-560. Zbl 1186.94017, MR 2602408, 10.1016/j.aml.2010.01.010 |
| Reference: | [28] Zhang, H., Hager, W. W.: A nonmonotone line search technique and its application to unconstrained optimization.SIAM. J. Optim. 14 (2004), 1043-1056. Zbl 1073.90024, MR 2112963, 10.1137/S1052623403428208 |
| Reference: | [29] Zhang, H., Ni, Q.: A new regularized quasi-Newton algorithm for unconstrained optimization.Appl. Math. Comput. 259 (2015), 460-469. Zbl 1390.90527, MR 3338382, 10.1016/j.amc.2015.02.032 |
| Reference: | [30] Zhang, H., Ni, Q.: A new regularized quasi-Newton method for unconstrained optimization.Optim. Lett. 12 (2018), 1639-1658. Zbl 1407.90313, MR 3857025, 10.1007/s11590-018-1236-z |
| Reference: | [31] Zheng, Y., Zheng, B.: A modified adaptive cubic regularization method for large-scale unconstrained optimization problem.Available at https://arxiv.org/abs/1904.07440 (2019), 21 pages. 10.48550/arXiv.1904.07440 |
| . |
Fulltext not available (moving wall 24 months)
Search
Partner of
