| Title:
|
Optimality conditions for maximizers of the information divergence from an exponential family (English) |
| Author:
|
Matúš, František |
| Language:
|
English |
| Journal:
|
Kybernetika |
| ISSN:
|
0023-5954 |
| Volume:
|
43 |
| Issue:
|
5 |
| Year:
|
2007 |
| Pages:
|
731-746 |
| Summary lang:
|
English |
| . |
| Category:
|
math |
| . |
| Summary:
|
The information divergence of a probability measure $P$ from an exponential family $\mathcal{E}$ over a finite set is defined as infimum of the divergences of $P$ from $Q$ subject to $Q\in \mathcal{E}$. All directional derivatives of the divergence from $\mathcal{E}$ are explicitly found. To this end, behaviour of the conjugate of a log-Laplace transform on the boundary of its domain is analysed. The first order conditions for $P$ to be a maximizer of the divergence from $\mathcal{E}$ are presented, including new ones when $P$ is not projectable to $\mathcal{E}$. (English) |
| Keyword:
|
Kullback–Leibler divergence |
| Keyword:
|
relative entropy |
| Keyword:
|
exponential family |
| Keyword:
|
information projection |
| Keyword:
|
log-Laplace transform |
| Keyword:
|
cumulant generating function |
| Keyword:
|
directional derivatives |
| Keyword:
|
first order optimality conditions |
| Keyword:
|
convex functions |
| Keyword:
|
polytopes |
| MSC:
|
52A20 |
| MSC:
|
60A10 |
| MSC:
|
62B10 |
| MSC:
|
90C90 |
| MSC:
|
94A17 |
| idZBL:
|
Zbl 1149.94007 |
| idMR:
|
MR2376334 |
| . |
| Date available:
|
2009-09-24T20:28:38Z |
| Last updated:
|
2012-06-06 |
| Stable URL:
|
http://hdl.handle.net/10338.dmlcz/135809 |
| . |
| Reference:
|
[1] Ay N.: An information-geometric approach to a theory of pragmatic structuring.Ann. Probab. 30 (2002), 416–436 Zbl 1010.62007, MR 1894113 |
| Reference:
|
[2] Ay N.: Locality of Global Stochastic Interaction in Directed Acyclic Networks.Neural Computation 14 (2002), 2959–2980 Zbl 1079.68582 |
| Reference:
|
[3] Ay N., Knauf A.: Maximizing multi-information.Kybernetika 45 (2006), 517–538 MR 2283503 |
| Reference:
|
[4] Ay N., Wennekers T.: Dynamical properties of strongly interacting Markov chains.Neural Networks 16 (2003), 1483–1497 |
| Reference:
|
[5] Barndorff-Nielsen O.: Information and Exponential Families in Statistical Theory.Wiley, New York 1978 Zbl 0387.62011, MR 0489333 |
| Reference:
|
[6] Brown L. D.: Fundamentals of Statistical Exponential Families.(Lecture Notes – Monograph Series 9.) Institute of Mathematical Statistics, Hayward, CA 1986 Zbl 0685.62002, MR 0882001 |
| Reference:
|
[8] Csiszár I., Matúš F.: Information projections revisited.IEEE Trans. Inform. Theory 49 (2003), 1474–1490 Zbl 1063.94016, MR 1984936 |
| Reference:
|
[9] Csiszár I., Matúš F.: Closures of exponential families.Ann. Probab. 33 (2005), 582–600 Zbl 1068.60008, MR 2123202 |
| Reference:
|
[10] Csiszár I., Matúš F.: Generalized maximum likelihood estimates for exponential families.To appear in Probab. Theory Related Fields (2008) Zbl 1133.62039, MR 2372970 |
| Reference:
|
[11] Pietra S. Della, Pietra, V. Della, Lafferty J.: Inducing features of random fields.IEEE Trans. Pattern Anal. Mach. Intell. 19 (1997), 380–393 |
| Reference:
|
[12] Letac G.: Lectures on Natural Exponential Families and their Variance Functions.(Monografias de Matemática 50.) Instituto de Matemática Pura e Aplicada, Rio de Janeiro 1992 Zbl 0983.62501, MR 1182991 |
| Reference:
|
[13] Matúš F.: Maximization of information divergences from binary i.i.d. sequences. In: Proc. IPMU 2004, Perugia 2004, Vol. 2, pp. 1303–1306 |
| Reference:
|
[14] Matúš F., Ay N.: On maximization of the information divergence from an exponential family.In: Proc. WUPES’03 (J. Vejnarová, ed.), University of Economics, Prague 2003, pp. 199–204 |
| Reference:
|
[15] Rockafellar R. T.: Convex Analysis.Princeton University Press, Priceton, N.J. 1970 MR 0274683 |
| Reference:
|
[16] Wennekers T., Ay N.: Finite state automata resulting from temporal information maximization.Theory in Biosciences 122 (2003), 5–18 Zbl 1090.68064 |
| . |
