Article
Full entry |
PDF
(0.9 MB)
Feedback
PDF
(0.9 MB)
Feedback
Keywords:
discrete-time Markov process; optimal stopping rule; stability index; total variation metric; contractive operator; optimal asset selling
discrete-time Markov process; optimal stopping rule; stability index; total variation metric; contractive operator; optimal asset selling
Summary:
We consider the optimal stopping problem for a discrete-time Markov process on a Borel state space $X$. It is supposed that an unknown transition probability $p(\cdot |x)$, $x\in X$, is approximated by the transition probability $\widetilde{p}(\cdot |x)$, $x\in X$, and the stopping rule $\widetilde{\tau }_*$, optimal for $\widetilde{p}$, is applied to the process governed by $p$. We found an upper bound for the difference between the total expected cost, resulting when applying $\widetilde{\tau }_*$, and the minimal total expected cost. The bound given is a constant times $\displaystyle \sup \nolimits _{x\in X}\Vert p(\cdot |x)-\widetilde{p}(\cdot |x)\Vert $, where $\Vert \cdot \Vert $ is the total variation norm.
References:
[1] Allart P.: Optimal stopping rules for correlated random walks with a discount. J. Appl. Prob. 41 (2004), 483–496 MR 2052586
[2] Bertsekas D. P.: Dynamic Programming: Deterministic and Stochastic Models. Prentice Hall, Englewood Cliffs, N. J. 1987 MR 0896902 | Zbl 0649.93001
[3] Bertsekas D. P., Shreve S. E.: Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York 1979 MR 0511544 | Zbl 0633.93001
[4] Dijk N. M. Van: Perturbation theory for unbounded Markov reward process with applications to queueing systems. Adv. in Appl. Probab. 20 (1988), 99–111 MR 0932536
[5] Dijk N. M. Van, Sladký K.: Error bounds for nonnegative dynamic models. J. Optim. Theory Appl. 101 (1999), 449–474 MR 1684679
[6] Dynkin E. B., Yushkevich A. A.: Controlled Markov Process. Springer-Verlag, New York 1979 MR 0554083
[7] Favero G., Runggaldier W. J.: A robustness results for stochastic control. Systems Control Lett. 46 (2002), 91–97 MR 2010062
[8] Gordienko E. I.: An estimate of the stability of optimal control of certain stochastic and deterministic systems. J. Soviet Math. 59 (1992), 891–899. (Translated from the Russian publication of 1989) MR 1163393
[9] Gordienko E. I., Salem F. S.: Robustness inequality for Markov control process with unbounded costs. Systems Control Lett. 33 (1998), 125–130 MR 1607814
[10] Gordienko E. I., Yushkevich A. A.: Stability estimates in the problem of average optimal switching of a Markov chain. Math. Methods Oper. Res. 57 (2003), 345–365 MR 1990916 | Zbl 1116.90401
[11] Gordienko E. I., Lemus-Rodríguez E., Montes-de-Oca R.: Discounted cost optimality problem: stability with respect to weak metrics. In press in: Math. Methhods Oper. Res. (2008) MR 2429561 | Zbl 1166.60041
[12] Hernández-Lerma O., Lassere J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, N.Y. 1996
[13] Jensen U.: An optimal stopping problem in risk theory. Scand. Actuarial J.2 (1997), 149–159 MR 1492423 | Zbl 0888.62104
[14] Meyn S. P., Tweedie R. L.: Markov Chains and Stochastic Stability. Springer-Verlag, London 1993 MR 1287609 | Zbl 1165.60001
[15] Montes-de-Oca R., Salem-Silva F.: Estimates for perturbations of an average Markov decision process with a minimal state and upper bounded by stochastically ordered Markov chains. Kybernetika 41 (2005), 757–772 MR 2193864
[16] Montes-de-Oca R., Sakhanenko, A., Salem-Silva F.: Estimate for perturbations of general discounted Markov control chains. Appl. Math. 30 (2003), 287–304 MR 2029538
[17] Muciek B. K.: Optimal stopping of a risk process: model with interest rates. J. Appl. Prob. 39 (2002), 261–270 MR 1908943 | Zbl 1011.62111
[18] Müller A.: How does the value function of a Markov decision process depend on the transition probabilities? Math. Oper. Res. 22 (1997), 872–885 MR 1484687
[19] Schäl M.: Conditions for optimality in dynamic programming and for the limit of $n$-stage optimal policies to be optimal. Z. Wahrsch. verw. Gebiete 32 (1975), 179–196 MR 0378841 | Zbl 0316.90080
[20] Shiryaev A. N.: Optimal Stopping Rules. Springer-Verlag, New York 1978 MR 2374974 | Zbl 1138.60008
[21] Shiryaev A. N.: Essential of Stochastic Finance. Facts, Models, Theory. World Scientific Publishing Co., Inc., River Edge, N.J. 1999 MR 1695318
Search
Partner of
