Article
Full entry |
PDF
(0.3 MB)
Feedback
PDF
(0.3 MB)
Feedback
Keywords:
partition of the state space; nonconstant optimal average cost; discounted approximations to the risk-sensitive average cost criterion; equality of superior and inferior limit risk-averse average criteria
partition of the state space; nonconstant optimal average cost; discounted approximations to the risk-sensitive average cost criterion; equality of superior and inferior limit risk-averse average criteria
Summary:
This work concerns controlled Markov chains with finite state space and compact action sets. The decision maker is risk-averse with constant risk-sensitivity, and the performance of a control policy is measured by the long-run average cost criterion. Under standard continuity-compactness conditions, it is shown that the (possibly non-constant) optimal value function is characterized by a system of optimality equations which allows to obtain an optimal stationary policy. Also, it is shown that the optimal superior and inferior limit average cost functions coincide.
References:
[1] A. Arapstathis, V. K. Borkar, E. Fernández-Gaucherand, M. K. Gosh, S. I. Marcus: Discrete-time controlled Markov processes with average cost criteria: a survey. SIAM J. Control Optim. 31 (1993), 282-334. DOI 10.1137/0331018 | MR 1205981
[2] P. Billingsley: Probability and Measure. Third edition. Wiley, New York 1995. MR 1324786 | Zbl 0822.60002
[3] R. Cavazos-Cadena, E. Fernández-Gaucherand: Controlled Markov chains with risk-sensitive criteria: average cost, optimality equations and optimal solutions. {Math. Method Optim. Res.} 43 (1999), 121-139. MR 1687362 | Zbl 0953.93077
[4] R. Cavazos-Cadena, E. Fernández-Gaucherand: Risk-sensitive control in communicating average Markov decision chains. In: { Modelling Uncertainty: An examination of Stochastic Theory, Methods and Applications} (M. Dror, P. L'Ecuyer and F. Szidarovsky, eds.), Kluwer, Boston 2002, pp. 525-544.
[5] R. Cavazos-Cadena: Solution to the risk-sensitive average cost optimality equation in a class of Markov decision processes with finite state space. {Math. Method Optim. Res.} 57 (2003), 263-285. DOI 10.1007/s001860200256 | MR 1973378 | Zbl 1023.90076
[6] R. Cavazos-Cadena, D. Hernández-Hernández: A characterization of the optimal risk-sensitive average cost in finite controlled Markov chains. {Ann. App. Probab.}, 15 (2005), 175-212. DOI 10.1214/105051604000000585 | MR 2115041 | Zbl 1076.93045
[7] R. Cavazos-Cadena, D. Hernández-Hernández: A system of Poisson equations for a non-constant Varadhan functional on a finite state space. {Appl. Math. Optim.} 53 (2006), 101-119. DOI 10.1007/s00245-005-0840-3 | MR 2190228
[8] R. Cavazos-Cadena, F. Salem-Silva: The discounted method and equivalence of average criteria for risk-sensitive Markov decision processes on Borel spaces. { Appl. Math. Optim.} 61 (2009), 167-190. DOI 10.1007/s00245-009-9080-2 | MR 2585141
[9] G. B. Di Masi, L. Stettner: Risk-sensitive control of discrete time Markov processes with infinite horizon. {SIAM J. Control Optim.} 38 1999, 61-78. DOI 10.1137/S0363012997320614 | MR 1740607 | Zbl 0946.93043
[10] G. B. Di Masi, L. Stettner: Infinite horizon risk sensitive control of discrete time Markov processes with small risk. {Syst. Control Lett.} 40 (2000), 15-20. DOI 10.1016/S0167-6911(99)00118-8 | MR 1829070 | Zbl 0977.93083
[11] G. B. Di Masi, L. Stettner: Infinite horizon risk sensitive control of discrete time Markov processes under minorization property. {SIAM J. Control Optim.} 46 (2007), 231-252. DOI 10.1137/040618631 | MR 2299627 | Zbl 1141.93067
[12] W. H. Fleming, W. M. McEneany: Risk-sensitive control on an infinite horizon. {SIAM J. Control Optim.} 33 (1995), 1881-1915. DOI 10.1137/S0363012993258720 | MR 1358100
[13] F. R. Gantmakher: The Theory of Matrices. {Chelsea}, London 1959.
[14] D. Hernández-Hernández, S. I. Marcus: Risk-sensitive control of Markov processes in countable state space. {Syst. Control Lett.} 29 (1996), 147-155. DOI 10.1016/S0167-6911(96)00051-5 | MR 1422212 | Zbl 0866.93101
[15] D. Hernández-Hernández, S. I. Marcus: Existence of risk sensitive optimal stationary policies for controlled Markov processes. {Appl. Math. Optim.} 40 (1999), 273-285. DOI 10.1007/s002459900126 | MR 1709324 | Zbl 0937.90115
[16] A. R. Howard, J. E. Matheson: Risk-sensitive Markov decision processes. {Management Sci.} 18 (1972), 356-369. DOI 10.1287/mnsc.18.7.356 | MR 0292497 | Zbl 0238.90007
[17] D. H. Jacobson: Optimal stochastic linear systems with exponential performance criteria and their relation to stochastic differential games. {IEEE Trans. Automat. Control} 18 (1973), 124-131. DOI 10.1109/TAC.1973.1100265 | MR 0441523
[18] S. C. Jaquette: Markov decison processes with a new optimality criterion: discrete time. {Ann. Statist.} 1 (1973), 496-505. DOI 10.1214/aos/1176342415 | MR 0378839
[19] S. C. Jaquette: A utility criterion for Markov decision processes. {Management Sci.} 23 (1976), 43-49. DOI 10.1287/mnsc.23.1.43 | MR 0439037 | Zbl 0337.90053
[20] A. Jaśkiewicz: Average optimality for risk sensitive control with general state space. {Ann. App. Probab.} 17 (2007), 654-675. DOI 10.1214/105051606000000790 | MR 2308338 | Zbl 1128.93056
[21] U. G. Rothblum, P. Whittle: Growth optimality for branching Markov decision chains. {Math. Oper. Res.} 7 (1982), 582-601. DOI 10.1287/moor.7.4.582 | MR 0686533 | Zbl 0498.90082
[22] K. Sladký: Successive approximation methods for dynamic programming models. In: Proc. Third Formator Symposium on the Analysis of Large-Scale Systems (J. Beneš and L. Bakule, eds.), Academia, Prague 1979, pp. 171-189. Zbl 0496.90081
[23] K. Sladký: Bounds on discrete dynamic programming recursions I. {Kybernetika} 16 (1980), 526-547. MR 0607292 | Zbl 0454.90085
[24] K. Sladký: Growth rates and average optimality in risk-sensitive Markov decision chains. {Kybernetika} 44 (2008), 205-226. MR 2428220 | Zbl 1154.90612
[25] K. Sladký, R. Montes-de-Oca: Risk-sensitive average optimality in Markov decision chains. In: Operations Research Proceedings, Vol. 2007, Part III (2008), pp. 69-74. DOI 10.1007/978-3-540-77903-2_11 | Zbl 1209.90348
[26] P. Whittle: Optimization Over Time-Dynamic Programming and Stochastic Control. Wiley, Chichester 1983. MR 0710833
[27] W. H. M. Zijm: Nonnegative Matrices in Dynamic Programming. Mathematical Centre Tract, Amsterdam 1983. MR 0723868 | Zbl 0526.90059
Search
Partner of
