[1] D. P. Bertsekas: Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Englewood Cliffs, N. J. 1987. MR 0896902 | Zbl 0649.93001

[2] D. P. Bertsekas, S. E. Shreve: Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York 1978. MR 0511544 | Zbl 0471.93002

[3] P. Billingsley: Convergence of Probability Measures. Wiley, New York 1968. MR 0233396 | Zbl 0172.21201

[4] D. Blackwell: Memoryless strategies in finite-stage dynamic programming. Ann. Math. Statist. 35 (1964), 863-865. MR 0162642 | Zbl 0127.36406

[5] D. Blackwell: Discounted dynamic programming. Ann. Math. Statist. 36 (1965), 226-235. MR 0173536 | Zbl 0133.42805

[6] V. S. Borkar: Control of Markov chains with long-run average cost criterion: the dynamic programming equations. SIAM J. Control Optim. 27 (1989), 642-657. MR 0993291 | Zbl 0668.60059

[7] R. Cavazos-Cadena: Solution to the optimality equation in a class of average Markov decision chains with unbounded costs. Kybernetika 27 (1991), 23-37. MR 1099512

[8] J. Diebolt, D. Guegan: Probabilistic properties of the general nonlinear markovian process of order one and applications to time series modelling. Rapport Technique No. 125, Laboratoire de Statistique Theorique et Appliquee, CNR-URA 1321, Universite Paris VI, 1990.

[9] J. L. Doob: Stochastic Processes. Wiley, New York 1953. MR 0058896 | Zbl 0053.26802

[10] M. Duflo: Methodes Recursives Aleatoires. Masson, Paris 1990. MR 1082344 | Zbl 0703.62084

[11] E. B. Dynkin, A. A. Yushkevich: Controlled Markov Processes. Springer - Verlag, Berlin 1979. MR 0554083

[12] R. Hartley: Dynamic programming and an undiscounted, infinite horizon, convex stochastic control problem. In: Recent Developments in Markov Decision Processes (R. Hartley, L. C. Thomas and D.J. White, eds.). Academic Press, London 1980, pp. 277-300.

[13] O. Hernandez-Lerma: Lyapunov criteria for stability of differential equations with Markov parameters. Boletin Soc. Mat. Mexicana 24 (1979), 27-48. MR 0579667 | Zbl 0486.60051

[14] O. Hernandez-Lerma: Adaptive Markov Control Processes. Springer - Verlag, New York 1989. MR 0995463 | Zbl 0698.90053

[15] O. Hernandez-Lerma: Average optimality in dynamic programming on Borel spaces - unbounded costs and controls. Syst. Control Lett. 17 (1991), 237-242. MR 1125975 | Zbl 0771.90098

[16] O. Hernandez-Lerma, J. B. Lasserre: Average cost optimal policies for Markov control processes with Borel state space and unbounded costs. Syst. Control Lett. 15 (1990), 349-356. MR 1078813 | Zbl 0723.93080

[17] O. Hernandez-Lerma, J. B. Lasserre: Linear programming and average optimality of Markov control processes on Borel spaces - unbounded costs. Rapport LAAS, LAAS-CNRS, Toulouse 1992. To appear in SIAM J. Control Optim. MR 1261150

[18] O. Hernandez-Lerma R. Montes de Oca, R. Cavazos-Cadena: Recurrence conditions for Markov decision processes with Borel state space: a survey. Ann. Oper. Res. 28 (1991), 29-46. MR 1105165

[19] K. Hinderer: Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter. Springer-Verlag, Berlin 1970. MR 0267890 | Zbl 0202.18401

[20] M. Yu. Kitayev: Semi-Markov and jump Markov control models: average cost criterion. Theory Probab. Appl. 30 (1985), 272-288. MR 0792619

[21] M. Kurano: The existence of a minimum pair of state and policy for Markov decision processes under the hypothesis of Doeblin. SIAM J. Control Optim. 27 (1989), 296-307. MR 0984830 | Zbl 0677.90085

[22] H. J. Kushner: Introduction to Stochastic Control. Holt, Rinehart and Winston, New York 1971. MR 0280248 | Zbl 0293.93018

[23] A. Leizarowitz: Optimal controls for diffusions in $R^n$. J. Math. Anal. Appl. 149 (1990), 180-209, MR 1054802

[24] S. P. Meyn: Ergodic theorems for discrete time stochastic systems using a stochastic Lyapunov function. SIAM J. Control Optim. 27 (1989), 1409-1439. MR 1022436 | Zbl 0681.60067

[25] A. Mokkadem: Sur un modele autoregressif nonlineaire. Ergodicite et ergodicite geometrique. J. Time Series Anal. 8 (1987), 195-205. MR 0886138

[26] D. Revuz: Markov Chains. Second edition. North-Holland, Amsterdam 1984. MR 0758799 | Zbl 0539.60073

[27] U. Rieder: Measurable selection theorems for optimization problems. Manuscripta Math. 24 (1978), 507-518. MR 0493590 | Zbl 0385.28005

[28] V. I. Rotar, T. A. Konyuhova: Two papers on asymptotic optimality in probability and almost surely. Preprint, Central Economic Mathematical Institute (CEMI), Moscow 1991.

[29] R. H. Stockbridge: Time-average control of martingale problems: a linear programming formulation. Ann. Probab. 18 (1990), 206-217. MR 1043944 | Zbl 0699.49019

[30] J. Wijngaard: Existence of average optimal strategies in markovian decision problems with strictly unbounded costs. In: Dynamic Programming and Its Applications (M. L. Puterman, ed.), Academic Press, New York 1978, pp. 369-386. MR 0537889 | Zbl 0458.90081

[31] K. Yosida: Functional Analysis. Fifth edition. Springer-Verlag, Berlin 1978. MR 0500055 | Zbl 0365.46001