| Title:
|
Discrete-time Markov control processes with recursive discount rates (English) |
| Author:
|
García, Yofre H. |
| Author:
|
González-Hernández, Juan |
| Language:
|
English |
| Journal:
|
Kybernetika |
| ISSN:
|
0023-5954 (print) |
| ISSN:
|
1805-949X (online) |
| Volume:
|
52 |
| Issue:
|
3 |
| Year:
|
2016 |
| Pages:
|
403-426 |
| Summary lang:
|
English |
| . |
| Category:
|
math |
| . |
| Summary:
|
This work analyzes a discrete-time Markov Control Model (MCM) on Borel spaces when the performance index is the expected total discounted cost. This criterion admits unbounded costs. It is assumed that the discount rate in any period is obtained by using recursive functions and a known initial discount rate. The classic dynamic programming method for finite-horizon case is verified. Under slight conditions, the existence of deterministic non-stationary optimal policies for infinite-horizon case is proven. Also, to find deterministic non-stationary $\epsilon-$optimal policies, the value-iteration method is used. To illustrate an example of recursive functions that generate discount rates, we consider the expected values of stochastic processes, which are solutions of certain class of Stochastic Differential Equations (SDE) between consecutive periods, when the initial condition is the previous discount rate. Finally, the consumption-investment problem and the discount linear-quadratic problem are presented as examples; in both cases, the discount rates are obtained using a SDE, similar to the Vasicek short-rate model. (English) |
| Keyword:
|
dynamic programming method |
| Keyword:
|
optimal stochastic control |
| MSC:
|
49L20 |
| MSC:
|
93E20 |
| idZBL:
|
Zbl 1357.49110 |
| idMR:
|
MR3532514 |
| DOI:
|
10.14736/kyb-2016-3-0403 |
| . |
| Date available:
|
2016-07-17T12:16:31Z |
| Last updated:
|
2023-08-22 |
| Stable URL:
|
http://hdl.handle.net/10338.dmlcz/145783 |
| . |
| Reference:
|
[1] Arnold, L.: Stochastic Differential Equations: Theory and Applications..John Wiley and Sons, New York 1973. Zbl 0278.60039, MR 0443083, 10.1002/zamm.19770570413 |
| Reference:
|
[2] Ash, R., Doléans-Dade, C.: Probability and Measure Theory..Academic Press, San Diego 2000. Zbl 0944.60004, MR 1810041 |
| Reference:
|
[3] Bellman, R.: Dynamic Programming..Princeton Univ. Press, New Jersey 1957. Zbl 1205.90002, MR 0090477 |
| Reference:
|
[4] Bertsekas, D., Shreve, S.: Stochastic Optimal Control: The Discrete Time Case..Athena Scientific, Massachusetts 1996. Zbl 0633.93001, MR 0511544 |
| Reference:
|
[5] Brigo, D., Mercurio, F.: Interest Rate Models Theory and Practice..Springer-Verlag, New York 2001. Zbl 1109.91023, MR 1846525, 10.1007/978-3-662-04553-4 |
| Reference:
|
[6] Black, F., Karasinski, P.: Bond and option pricing when short rates are lognormal..Financ. Anal. J. 47 (1991), 4, 52-59. 10.2469/faj.v47.n4.52 |
| Reference:
|
[7] Carmon, Y., Shwartz, A.: Markov decision processes with exponentially representable discounting..Oper. Res. Lett. 37 (2009), 51-55. Zbl 1154.90610, MR 2488083, 10.1016/j.orl.2008.10.005 |
| Reference:
|
[8] Vecchia, E. Della, Marco, S. Di, Vidal, F.: Dynamic programming for variable discounted Markov decision problems..In: Jornadas Argentinas de Informática e Investigación Operativa (43JAIIO) - XII Simposio Argentino de Investigación Operativa (SIO), Buenos Aires, 2014, pp. 50-62. |
| Reference:
|
[9] Cox, J., Ingersoll, J., Ross, S.: A theory of the term structure of interest rates..Econometrica 53 (1985), 385-407. Zbl 1274.91447, MR 0785475, 10.2307/1911242 |
| Reference:
|
[10] Dothan, U.: On the term structure interest rates..J. Financ. Econ. 6 (1978), 59-69. 10.1016/0304-405x(78)90020-x |
| Reference:
|
[11] Feinberg, E., Shwartz, A.: Markov decision models with weighted discounted criteria..J. Finan. Econ. 19 (1994), 152-168. Zbl 0803.90123, MR 1290017, 10.1287/moor.19.1.152 |
| Reference:
|
[12] González-Hernández, J., López-Martínez, R., Pérez-Hernández, J.: Markov control processes with randomized discounted cost..Math. Method Oper. Res. 65 (2006), 1, 27-44. Zbl 1126.90075, MR 2302022, 10.1007/s00186-006-0092-2 |
| Reference:
|
[13] González-Hernández, J., López-Martínez, R., Minjarez-Sosa, A.: Adaptive policies for stochastic systems under a randomized discounted cost criterion..Bol. Soc. Mat. Mex. 14 (2008), 3, 149-163. Zbl 1201.93130, MR 2667162 |
| Reference:
|
[14] González-Hernández, J., López-Martínez, R., Minjarez-Sosa, A.: Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion..Kybernetika 45 (2008), 5, 737-754. Zbl 1190.93105, MR 2599109 |
| Reference:
|
[15] Guo, X., Hernández-Del-Valle, A., Hernández-Lerma, O.: First passage problems for a non-stationary discrette-time stochastic control systems..Eur. J. Control 15 (2012), 7, 528-538. MR 3086896, 10.3166/ejc.18.528-538 |
| Reference:
|
[16] Hernández-Lerma, O., Lasserre, J. B.: Discrete-Time Markov Control Processes. Basic Optimality Criteria..Springer-Verlag, New York 1996. Zbl 0840.93001, MR 1363487, 10.1007/978-1-4612-0729-0 |
| Reference:
|
[17] Minjarez-Sosa, J.: Markov control models with unknown random sate-action-dependent discount factors..TOP 23 (2015), 3, 743-772. MR 3407674, 10.1007/s11750-015-0360-5 |
| Reference:
|
[18] Hinderer, K.: Foundations of non-stationary dynamical programming with discrete time parameter..In: Lecture Notes Operations Research (M. Bechmann and H. Künzi, eds.), Springer-Verlag 33, Zürich 1970. MR 0267890, 10.1007/978-3-642-46229-0 |
| Reference:
|
[19] Ho, T., Lee, S.: Term structure movements and pricing interest rate claims..J. Financ. 41 (1986), 1011-1029. 10.1111/j.1540-6261.1986.tb02528.x |
| Reference:
|
[20] Hull, J.: Options, Futures and other Derivatives. Sixth edition..Prentice Hall, New Jersey 2006. |
| Reference:
|
[21] Hull, J., White, A.: Pricing interest rate derivative securities..Rev. Financ. Stud. 3 (1990), 573-592. 10.1093/rfs/3.4.573 |
| Reference:
|
[22] Mercurio, F., Moraleda, J.: A family of humped volatility models..Eur. J. Finance 7 (2001), 93-116. 10.1080/13518470151141440 |
| Reference:
|
[23] Rendleman, R., Bartter, B.: The pricing of options on debt securities..J. Financ. Quant. Anal. 15 (1980), 11-24. 10.2307/2979016 |
| Reference:
|
[24] Rieder, U.: Measurable selection theorems for optimization problems..Manuscripta Math. 24 (1978), 115-131. Zbl 0385.28005, MR 0493590, 10.1007/bf01168566 |
| Reference:
|
[25] Schäl, M.: Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal..Z. Wahrscheinlichkeit 32 (1975), 179-196. Zbl 0316.90080, MR 0378841, 10.1007/bf00532612 |
| Reference:
|
[26] Vasicek, O.: An equilibrium characterization of the term structure..J. Financ. Econ. 5 (1977), 177-188. 10.1016/0304-405x(77)90016-2 |
| Reference:
|
[27] Wei, Q., X, X. Guo: Markov decision processes with state-dependent discount factors and unbounded rewards costs..Oper. Res. Lett. 39 (2011), 369-374. Zbl 1235.90178, MR 2835530, 10.1016/j.orl.2011.06.014 |
| Reference:
|
[28] Ye, L., Guo, X.: Continuous-time Markov decision processes with state-dependent discount factors..Acta Appl. Math. 121 (2012), 1, 5-27. Zbl 1281.90082, MR 2966962, 10.1007/s10440-012-9669-3 |
| Reference:
|
[29] Zhang, Y.: Convex analytic approach to constrained discounted Markov decision processes with non-constant discount factors..TOP 21 (2013), 2, 378-408. Zbl 1273.90235, MR 3068494, 10.1007/s11750-011-0186-8 |
| . |
