Previous |  Up |  Next


Title: Time-discretization for controlled Markov processes. I. General approximation results (English)
Author: van Dijk, Nico M.
Author: Hordijk, Arie
Language: English
Journal: Kybernetika
ISSN: 0023-5954
Volume: 32
Issue: 1
Year: 1996
Pages: 1-16
Category: math
MSC: 49K45
MSC: 49M25
MSC: 90C40
MSC: 93C57
MSC: 93E20
idZBL: Zbl 0874.93094
idMR: MR1380195
Date available: 2009-09-24T19:00:07Z
Last updated: 2012-06-06
Stable URL:
Related article:
Reference: [1] A. Bensoussan M. Robin: On the convergence of the discrete time dynamic programming equation for general semi-groups.SIAM J. Control Optim. 20 (1982), 1, 722-746. MR 0667651
Reference: [2] N. Christopeit: Discrete approximation of continuous time stochastic control systems.SIAM J. Control Optim. 21 (1983), 1, 17-40. Zbl 0508.93065, MR 0688438
Reference: [3] B. T. Doshi: Optimal control of the service rate in an $M|G|1$-queueing system.Adv. in Appl. Probab. 10 (1978), 682-701. Zbl 0381.60086, MR 0499221
Reference: [4] W. H. Fleming R. W. Rishel: Deterministic and Stochastic Optimal Control.Springer Verlag, Berlin 1975. MR 0454768
Reference: [5] I. I. Gihman A. V. Skorohod: Controlled Stochastic Processes.Springer Verlag, Berlin 1979. MR 0544839
Reference: [6] U. G. Haussmann: A discrete approximation to optimal stochastic.In: Analysis and Optimization of Stochastic Systems, Academic Press, London 1980, pp. 229-241. Zbl 0476.93082, MR 0592987
Reference: [7] A. Hordijk F. A. Van der Duyn Schouten: Average optimal policies in Markov decision drift processes with applications to queueing and replacement model.Adv. in Appl. Probab. 15 (1983), 274-303. MR 0698820
Reference: [8] A. Hordijk F. A. Van der Duyn Schouten: Discretization and weak convergence in Markov decision drift processes.Math. Oper. Res. 9 (1984), 1, 112-141. MR 0736642
Reference: [9] A. Hordijk F. A. Van der Duyn Schouten: Markov decision drift processes; Conditions for optimality obtained by discretization.Math. Oper. Res. 10 (1985), 160-173. MR 0787013
Reference: [10] A. Hordijk F. A. Van der Duyn Schouten: On the optimality of $(s,S)$-policies in continuous review inventory models.SIAM J. Appl. Math. 46 (1986), 912-929. MR 0859000
Reference: [11] G. M. Koole: Stochastic Scheduling and Dynamic Programming.Ph.D. Thesis, University of Leiden 1992.
Reference: [12] T. G. Kurtz: Extensions of Trotter's operator semigroup approximations theorems.J. Funct. Anal. 3 (1969), 111-132. MR 0242016
Reference: [13] H. J. Kushner: Probability Methods for Approximation in Stochastic Control and for Elliptic Equations.Academic Press, New York 1977. MR 0469468
Reference: [14] P. D. Lax R. D. Richtmeyer: Survey of the stability of linear finite difference equations.Comm. Pure Appl. Math. 9 (1956), 267-293. MR 0079204
Reference: [15] T. Meis U. Marcowitz: Numerical Solution of Partial Differential Equations.Springer Verlag, Berlin 1981. MR 0617910
Reference: [16] H. J. Plum: Impulsive and continuously acting control of jump processes -- Time discretization.Stochastics and Stochastic Reports 36 (1991), 163-192. Zbl 0739.60076, MR 1128492
Reference: [17] R. Rishel: Necessary and sufficient dynamic programming conditions for continuous time stochastic optimal control.SIAM J. Control 8 (1970), 4, 559-571. Zbl 0206.45804, MR 0274161
Reference: [18] R. Rishel: Controls optimal from the toward and dynamic programming for systems of controlled jump processes.Math. Programming Study 6 (1976), 125-153. MR 0479611
Reference: [19] F. A. Van der Duyn Schouten: Markov Decision Processes with Continuous Time Parameter.Mathematical Centre Tract 164, Amsterdam 1983. Zbl 0519.90052, MR 0709095
Reference: [20] N. M. Van Dijk: Controlled Markov Processes; Time Discretization/Networks of Queues.Ph.D. Thesis, University of Leiden 1983.
Reference: [21] N. M. Van Dijk: On the finite horizon Bellman equation for controlled Markov jump models with unbounded characteristics: existence and approximations.Stochastic Process. Appl. 28 (1988), 141-157. MR 0936380


Files Size Format View
Kybernetika_32-1996-1_1.pdf 1.391Mb application/pdf View/Open
Back to standard record
Partner of
EuDML logo