Article
| Title: | First- and second-order adjoint methods for stochastic identification problems (English) |
| Author: | Anh, Nguyen Thi Van |
| Author: | Heldt, Adrian |
| Author: | Khan, Akhtar Ali |
| Author: | Tammer, Christiane |
| Language: | English |
| Journal: | Applications of Mathematics |
| ISSN: | 0862-7940 (print) |
| ISSN: | 1572-9109 (online) |
| Volume: | 70 |
| Issue: | 6 |
| Year: | 2025 |
| Pages: | 763-795 |
| Summary lang: | English |
| . | |
| Category: | math |
| . | |
| Summary: | We present a unified framework for estimating stochastic parameters in general variational problems. This nonlinear inverse problem is formulated as a stochastic optimization problem using the output least-squares (OLS) objective, which minimizes the discrepancy between observed data and the computed solution. A key challenge in OLS-based formulations is the efficient computation of first- and second-order derivatives of the OLS functional, which depend on the corresponding derivatives of the parameter-to-solution map often costly and difficult to evaluate, especially in stochastic settings. To address this, we develop a rigorous computational approach based on first- and second-order adjoint methods for inverse problems governed by stochastic variational problems. Specifically, we propose a new first-order adjoint method for computing the gradient of the OLS objective and introduce two novel second-order adjoint methods for Hessian evaluation. A stochastic Galerkin discretization framework is employed, enabling efficient implementation of the adjoint-based derivative computations. Numerical experiments demonstrate the accuracy and efficiency of the proposed computational framework. (English) |
| Keyword: | stochastic inverse problem |
| Keyword: | partial differential equations with random data |
| Keyword: | stochastic Galerkin method |
| Keyword: | regularization |
| Keyword: | first-order adjoint method |
| Keyword: | second-order adjoint method |
| MSC: | 35R30 |
| MSC: | 49N45 |
| MSC: | 65J20 |
| MSC: | 65J22 |
| MSC: | 65M30 |
| DOI: | 10.21136/AM.2025.0151-25 |
| . | |
| Date available: | 2025-12-20T05:03:05Z |
| Last updated: | 2025-12-22 |
| Stable URL: | http://hdl.handle.net/10338.dmlcz/153223 |
| . | |
| Reference: | [1] Alekseev, A. K., Navon, I. M.: The analysis of an ill-posed problem using multi-scale resolution and second-order adjoint techniques.Comput. Methods Appl. Mech. Eng. 190 (2001), 1937-1953. Zbl 1031.76040, MR 1813303, 10.1016/S0045-7825(00)00299-1 |
| Reference: | [2] Alekseev, A. K., Navon, I. M., Steward, J. L.: Comparison of advanced large-scale minimization algorithms for the solution of inverse ill-posed problems.Optim. Methods Softw. 24 (2009), 63-87. Zbl 1189.90221, MR 2489430, 10.1080/10556780802370746 |
| Reference: | [3] Alexanderian, A., Petra, N., Stadler, G., Ghattas, O.: Mean-variance risk-averse optimal control of systems governed by PDEs with random parameter fields using quadratic approximations.SIAM/ASA J. Uncertain. Quantif. 5 (2017), 1166-1192. Zbl 1391.93289, MR 3725286, 10.1137/16M106306X |
| Reference: | [4] Altaf, M. U., Gharamti, M. El, Heemink, A. W., Hoteit, I.: A reduced adjoint approach to variational data assimilation.Comput. Methods Appl. Mech. Eng. 254 (2013), 1-13. Zbl 1297.93053, MR 3002758, 10.1016/j.cma.2012.10.003 |
| Reference: | [5] Arens, K., Rentrop, P., Stoll, S. O., Wever, U.: An adjoint approach to optimal design of turbine blades.Appl. Numer. Math. 53 (2005), 93-105. Zbl 1116.49326, MR 2128516, 10.1016/j.apnum.2004.11.003 |
| Reference: | [6] Narayanan, V. A. Badri, Zabaras, N.: Stochastic inverse heat conduction using a spectral approach.Int. J. Numer. Methods Eng. 60 (2004), 1569-1593. Zbl 1098.80008, MR 2068927, 10.1002/nme.1015 |
| Reference: | [7] Benner, P., Onwunta, A., Stoll, M.: Block-diagonal preconditioning for optimal control problems constrained by PDEs with uncertain inputs.SIAM J. Matrix Anal. Appl. 37 (2016), 491-518. Zbl 1382.65074, MR 3483160, 10.1137/15M1018502 |
| Reference: | [8] Boger, D. A., Paterson, E. G.: A continuous adjoint approach to design optimization in cavitating flow using a barotropic model.Comput. Fluids 101 (2014), 155-169. Zbl 1391.76062, MR 3233426, 10.1016/j.compfluid.2014.06.014 |
| Reference: | [9] Borggaard, J., Wyk, H.-W. van: Gradient-based estimation of uncertain parameters for elliptic partial differential equations.Inverse Probl. 31 (2015), Article ID 065008, 33 pages. Zbl 1321.35028, MR 3350627, 10.1088/0266-5611/31/6/065008 |
| Reference: | [10] Borzì, A., Winckel, G. von: Multigrid methods and sparse-grid collocation techniques for parabolic optimal control problems with random coefficients.SIAM J. Sci. Comput. 31 (2009), 2172-2192. Zbl 1196.35029, MR 2516148, 10.1137/070711311 |
| Reference: | [11] Cacuci, D. G.: Second-order adjoint sensitivity analysis methodology (2nd-ASAM) for computing exactly and efficiently first- and second-order sensitivities in large-scale linear systems. I. Computational methodology.J. Comput. Phys. 284 (2015), 687-699. Zbl 1352.65130, MR 3303639, 10.1016/j.jcp.2014.12.042 |
| Reference: | [12] Cacuci, D. G.: Second-order adjoint sensitivity analysis methodology (2nd-ASAM) for computing exactly and efficiently first- and second-order sensitivities in large-scale linear systems. II. Illustrative application to a paradigm particle diffusion problem.J. Comput. Phys. 284 (2015), 700-717. Zbl 1352.65131, MR 3303640, 10.1016/j.jcp.2014.11.030 |
| Reference: | [13] Cahill, N. D., Jadamba, B., Khan, A. A., Sama, M., Winkler, B. C.: A first-order adjoint and a second-order hybrid method for an energy output least-squares elastography inverse problem of identifying tumor location.Bound. Value Probl. 2013 (2013), Article ID 263, 19 pages. Zbl 1292.35323, MR 3341371, 10.1186/1687-2770-2013-263 |
| Reference: | [14] Chen, P., Quarteroni, A., Rozza, G.: Stochastic optimal Robin boundary control problems of advection-dominated elliptic equations.SIAM J. Numer. Anal. 51 (2013), 2700-2722. Zbl 1281.49015, MR 3115461, 10.1137/120884158 |
| Reference: | [15] Chen, P., Quarteroni, A., Rozza, G.: Multilevel and weighted reduced basis method for stochastic optimal control problems constrained by Stokes equations.Numer. Math. 133 (2016), 67-102. Zbl 1344.93109, MR 3475656, 10.1007/s00211-015-0743-4 |
| Reference: | [16] Chen, Z., Zhang, W., Zou, J.: Stochastic convergence of regularized solutions and their finite element approximations to inverse source problems.SIAM J. Numer. Anal. 60 (2022), 751-780. Zbl 07506907, MR 4403550, 10.1137/21M1409779 |
| Reference: | [17] Cho, M., Jadamba, B., Kahler, R., Khan, A. A., Sama, M.: First-order and second-order adjoint methods for the inverse problem of identifying non-linear parameters in PDEs.Industrial Mathematics and Complex Systems Springer, Singapore (2017), 147-163. Zbl 1396.49028, MR 3728376, 10.1007/978-981-10-3758-0_9 |
| Reference: | [18] Cho, M., Jadamba, B., Khan, A. A., Oberai, A. A., Sama, M.: Identification in mixed variational problems by adjoint methods with applications.Modeling and Optimization: Theory and Applications Springer Proceedings in Mathematics & Statistics 213. Springer, Cham (2017), 65-84. Zbl 1384.49009, MR 3746179, 10.1007/978-3-319-66616-7_5 |
| Reference: | [19] Choi, Y., Lee, H.-C.: Error analysis of finite element approximations of the optimal control problem for stochastic Stokes equations with additive white noise.Appl. Numer. Math. 133 (2018), 144-160. Zbl 1397.65255, MR 3843759, 10.1016/j.apnum.2018.03.002 |
| Reference: | [20] Cioaca, A., Alexe, M., Sandu, A.: Second-order adjoints for solving PDE-constrained optimization problems.Optim. Methods Softw. 27 (2012), 625-653. Zbl 1260.49060, MR 2946050, 10.1080/10556788.2011.610455 |
| Reference: | [21] Cioaca, A., Sandu, A.: An optimization framework to improve 4D-var data assimilation system performance.J. Comput. Phys. 275 (2014), 377-389. Zbl 1349.90887, MR 3250338, 10.1016/j.jcp.2014.07.005 |
| Reference: | [22] Daescu, D. N., Navon, I. M.: Efficiency of a POD-based reduced second-order adjoint model in 4D-var data assimilation.Int. J. Numer. Methods Fluids 53 (2007), 985-1004. Zbl 1370.76122, MR 2289211, 10.1002/fld.1316 |
| Reference: | [23] Dambrine, M., Khan, A. A., Sama, M.: A stochastic regularized second-order iterative scheme for optimal control and inverse problems in stochastic partial differential equations.Philos. Trans. R. Soc. Lond., A, Math. Phys. Eng. Sci. 380 (2022), Article ID 20210352, 13 pages. MR 4505744, 10.1098/rsta.2021.0352 |
| Reference: | [24] Dambrine, M., Khan, A. A., Sama, M., Starkloff, H.-J.: Stochastic elliptic inverse problems: Solvability, convergence rates, discretization, and applications.J. Convex Anal. 30 (2023), 851-885. Zbl 1522.35577, MR 4650286 |
| Reference: | [25] Dippon, J., Gwinner, J., Khan, A. A., Sama, M.: A new regularized stochastic approximation framework for stochastic inverse problems.Nonlinear Anal., Real World Appl. 73 (2023), Article ID 103869, 29 pages. Zbl 1519.35367, MR 4563581, 10.1016/j.nonrwa.2023.103869 |
| Reference: | [26] Dominguez, N., Gibiat, V., Esquerre, Y.: Time domain topological gradient and time reversal analogy: An inverse method for ultrasonic target detection.Wave Motion 42 (2005), 31-52. Zbl 1189.74070, MR 2133087, 10.1016/j.wavemoti.2004.09.005 |
| Reference: | [27] Faverjon, B., Puig, B., Baranger, T. N.: Identification of boundary conditions by solving Cauchy problem in linear elasticity with material uncertainties.Comput. Math. Appl. 73 (2017), 494-504. Zbl 1370.35275, MR 3600206, 10.1016/j.camwa.2016.12.011 |
| Reference: | [28] Gong, B., Liu, W., Tang, T., Zhao, W., Zhou, T.: An efficient gradient projection method for stochastic optimal control problems.SIAM J. Numer. Anal. 55 (2017), 2982-3005. Zbl 1386.60239, MR 3730543, 10.1137/17M1123559 |
| Reference: | [29] Gunzburger, M. D., Lee, H.-C., Lee, J.: Error estimates of stochastic optimal Neumann boundary control problems.SIAM J. Numer. Anal. 49 (2011), 1532-1552. Zbl 1243.65008, MR 2831060, 10.1137/100801731 |
| Reference: | [30] Guth, P. A., Kaarnioja, V., Kuo, F. Y., Schillings, C., Sloan, I. H.: A quasi-Monte Carlo method for optimal control under uncertainty.SIAM/ASA J. Uncertain. Quantif. 9 (2021), 354-383. Zbl 1475.49004, MR 4241521, 10.1137/19M1294952 |
| Reference: | [31] Hawks, R., Jadamba, B., Khan, A. A., Sama, M., Yang, Y.: A variational inequality based stochastic approximation for inverse problems in stochastic partial differential equations.Nonlinear Analysis and Global Optimization Springer Optimization and Its Applications 167. Springer, Cham (2021), 207-226. Zbl 1472.35451, MR 4238578, 10.1007/978-3-030-61732-5_9 |
| Reference: | [32] Huang, F., Chen, Y., Chen, Y., Sun, H.: Stochastic collocation for optimal control problems with stochastic PDE constraints by meshless techniques.J. Math. Anal. Appl. 530 (2024), Article ID 127634, 18 pages. Zbl 07759006, MR 4623961, 10.1016/j.jmaa.2023.127634 |
| Reference: | [33] Jadamba, B., Khan, A. A., Oberai, A. A., Sama, M.: First-order and second-order adjoint methods for parameter identification problems with an application to the elasticity imaging inverse problem.Inverse Probl. Sci. Eng. 25 (2017), 1768-1787. Zbl 1387.92057, MR 3698480, 10.1080/17415977.2017.1289195 |
| Reference: | [34] Jadamba, B., Khan, A. A., Raciti, F., Sama, M.: A variational inequality based stochastic approximation for estimating the flexural rigidity in random fourth-order models.Commun. Nonlinear Sci. Numer. Simul. 111 (2022), Article ID 106406, 11 pages. Zbl 1489.35312, MR 4402577, 10.1016/j.cnsns.2022.106406 |
| Reference: | [35] Jadamba, B., Khan, A. A., Sama, M.: A regularized stochastic subgradient projection method for an optimal control problem in a stochastic partial differential equation.Mathematical Analysis in Interdisciplinary Research Springer Optimization and Its Applications 179. Springer, Cham (2021), 417-429. Zbl 1500.93146, MR 4404289, 10.1007/978-3-030-84721-0_19 |
| Reference: | [36] Jadamba, B., Khan, A. A., Sama, M., Starkloff, H.-J., Tammer, C.: A convex optimization framework for the inverse problem of identifying a random parameter in a stochastic partial differential equation.SIAM/ASA J. Uncertain. Quantif. 9 (2021), 922-952. Zbl 1471.35332, MR 4279154, 10.1137/20M1323953 |
| Reference: | [37] Jadamba, B., Khan, A. A., Sama, M., Yang, Y.: An iteratively regularized stochastic gradient method for estimating a random parameter in a stochastic PDEs: A variational inequality approach.J. Nonlinear Var. Anal. 5 (2021), 865-880. Zbl 07557729, 10.23952/jnva.5.2021.6.02 |
| Reference: | [38] Jensen, J. S., Nakshatrala, P. B., Tortorelli, D. A.: On the consistency of adjoint sensitivity analysis for structural optimization of linear dynamic problems.Struct. Multidiscip. Optim. 49 (2014), 831-837. MR 3198981, 10.1007/s00158-013-1024-4 |
| Reference: | [39] Jin, B., Zou, J.: Inversion of Robin coefficient by a spectral stochastic finite element approach.J. Comput. Phys. 227 (2008), 3282-3306. Zbl 1141.65005, MR 2392734, 10.1016/j.jcp.2007.11.042 |
| Reference: | [40] Kennedy, G. J., Hansen, J. S.: The hybrid-adjoint method: A semi-analytic gradient evaluation technique applied to composite cure cycle optimization.Optim. Eng. 11 (2010), 23-43. Zbl 1273.74389, MR 2601731, 10.1007/s11081-008-9068-9 |
| Reference: | [41] Keyanpour, M., Nehrani, A. M.: Optimal thickness of a cylindrical shell subject to stochastic forces.J. Optim. Theory Appl. 167 (2015), 1032-1050. Zbl 1332.74043, MR 3424702, 10.1007/s10957-014-0663-y |
| Reference: | [42] Knopoff, D. A., Fernández, D. R., Torres, G. A., Turner, C. V.: Adjoint method for a tumor growth PDE-constrained optimization problem.Comput. Math. Appl. 66 (2013), 1104-1119. Zbl 07863342, MR 3093483, 10.1016/j.camwa.2013.05.028 |
| Reference: | [43] Kourounis, D., Durlofsky, L. J., Jansen, J. D., Aziz, K.: Adjoint formulation and constraint handling for gradient-based optimization of compositional reservoir flow.Comput. Geosci. 18 (2014), 117-137. Zbl 1393.76122, MR 3208745, 10.1007/s10596-013-9385-8 |
| Reference: | [44] Kubínová, M., Pultarová, I.: Block preconditioning of stochastic Galerkin problems: New two-sided guaranteed spectral bounds.SIAM/ASA J. Uncertain. Quantif. 8 (2020), 88-113. Zbl 1447.65151, MR 4051982, 10.1137/19M125902X |
| Reference: | [45] Dimet, F.-X. Le, Navon, I. M., Daescu, D. N.: Second-order information in data assimilation.Mon. Weather Rev. 130 (2002), 629-648. 10.1175/1520-0493(2002)130<0629:SOIIDA>2.0.CO;2 |
| Reference: | [46] Lewis, R. M.: Numerical computation of sensitivities and the adjoint approach.Computational Methods for Optimal Design and Control Progress in Systems and Control Theory 24. Birkhäuser Boston (1998), 285-302. Zbl 0924.65054, MR 1646782, 10.1007/978-1-4612-1780-0_16 |
| Reference: | [47] Liu, G., Geier, M., Liu, Z., Krafczyk, M., Chen, T.: Discrete adjoint sensitivity analysis for fluid flow topology optimization based on the generalized lattice Boltzmann method.Comput. Math. Appl. 68 (2014), 1374-1392. Zbl 1367.76044, MR 3272547, 10.1016/j.camwa.2014.09.002 |
| Reference: | [48] Lord, G. J., Powell, C. E., Shardlow, T.: An Introduction to Computational Stochastic PDEs.Cambridge Texts in Applied Mathematics. Cambridge University Press, New York (2014). Zbl 1327.60011, MR 3308418, 10.1017/CBO9781139017329 |
| Reference: | [49] Lozano, C.: Discrete surprises in the computation of sensitivities from boundary integrals in the continuous adjoint approach to inviscid aerodynamic shape optimization.Comput. Fluids 56 (2012), 118-127. Zbl 1365.76100, MR 2978800, 10.1016/j.compfluid.2011.12.001 |
| Reference: | [50] Zanganeh, M. Namdar, Kraaijevanger, J. F. B. M., Buurman, H. W., Jansen, J. D., Rossen, W. R.: Challenges in adjoint-based optimization of a foam EOR process.Comput. Geosci. 18 (2014), 563-577. MR 3253767, 10.1007/s10596-014-9412-4 |
| Reference: | [51] Nouy, A., Soize, C.: Random field representations for stochastic elliptic boundary value problems and statistical inverse problems.Eur. J. Appl. Math. 25 (2014), 339-373. Zbl 1298.60056, MR 3197997, 10.1017/S0956792514000072 |
| Reference: | [52] Oberai, A. A., Gokhale, N. H., Feijóo, G. R.: Solution of inverse problems in elasticity imaging using the adjoint method.Inverse Probl. 19 (2003), 297-313. Zbl 1171.35490, MR 1991790, 10.1088/0266-5611/19/2/304 |
| Reference: | [53] Papadimitriou, D. I., Giannakoglou, K. C.: Aerodynamic shape optimization using first and second order adjoint and direct approaches.Arch. Comput. Methods Eng. 15 (2008), 447-488. Zbl 1170.76348, MR 2453310, 10.1007/s11831-008-9025-y |
| Reference: | [54] Pingen, G., Evgrafov, A., Maute, K.: Adjoint parameter sensitivity analysis for the hydrodynamic lattice Boltzmann method with applications to design optimization.Comput. Fluids 38 (2009), 910-923. Zbl 1242.76254, MR 2645691, 10.1016/j.compfluid.2008.10.002 |
| Reference: | [55] Reséndiz, E., Pinnau, R.: Adjoint-based optimization of particle trajectories in laminar flows.Appl. Math. Comput. 248 (2014), 567-583. Zbl 1338.76027, MR 3276711, 10.1016/j.amc.2014.10.007 |
| Reference: | [56] Shen, W., Ge, L., Liu, W.: Stochastic Galerkin method for optimal control problem governed by random elliptic PDE with state constraints.J. Sci. Comput. 78 (2019), 1571-1600. Zbl 1428.65095, MR 3934678, 10.1007/s10915-018-0823-6 |
| Reference: | [57] Tottorelli, D. A., Michaleris, P.: Design sensitivity analysis: Overview and review.Inverse Probl. Eng. 1 (1994), 71-105. 10.1080/174159794088027573 |
| Reference: | [58] Touzeau, C., Magnain, B., Lubineau, G., Florentin, E.: Robust method for identifying material parameters based on virtual fields in elastodynamics.Comput. Math. Appl. 77 (2019), 3021-3042. Zbl 1442.74077, MR 3958649, 10.1016/j.camwa.2018.08.005 |
| Reference: | [59] Uy, W. I. T., Grigoriu, M.: Specification of additional information for solving stochastic inverse problems.SIAM J. Sci. Comput. 41 (2019), A2880--A2910. Zbl 1439.35571, MR 4008643, 10.1137/18M120155X |
| Reference: | [60] Volkov, O., Protas, B., Liao, W., Glander, D. W.: Adjoint-based optimization of thermo-fluid phenomena in welding processes.J. Eng. Math. 65 (2009), 201-220. Zbl 1180.76050, MR 2546438, 10.1007/s10665-009-9292-0 |
| Reference: | [61] Wang, S.-L., Yang, Y.-F., Zeng, Y.-H.: The adjoint method for the inverse problem of option pricing.Math. Probl. Eng. 2014 (2014), Article ID 314104, 7 pages. Zbl 1407.91275, MR 3191166, 10.1155/2014/314104 |
| Reference: | [62] Wang, Y., Chen, P., Li, W.: Projected Wasserstein gradient descent for high-dimensional Bayesian inference.SIAM/ASA J. Uncertain. Quantif. 10 (2022), 1513-1532. Zbl 1506.62270, MR 4512980, 10.1137/21M1454018 |
| Reference: | [63] Wang, Z., Navon, I. M., Dimet, F. X. Le, Zou, X.: The second order adjoint analysis: Theory and applications.Meteorology Atmospheric Phys. 50 (1992), 3-20. 10.1007/BF01025501 |
| Reference: | [64] Ye, X., Li, P., Liu, F. Y.: Exact time-domain second-order adjoint-sensitivity computation for linear circuit analysis and optimization.IEEE Trans. Circuits Syst. I, Regul. Pap. 57 (2010), 236-248. Zbl 1469.94232, MR 2729824, 10.1109/TCSI.2009.2015720 |
| . |
Fulltext not available (moving wall 24 months)
Search
Partner of
