| Title:
|
Maximizing the Bregman divergence from a Bregman family (English) |
| Author:
|
Rauh, Johannes |
| Author:
|
Matúš, František |
| Language:
|
English |
| Journal:
|
Kybernetika |
| ISSN:
|
0023-5954 (print) |
| ISSN:
|
1805-949X (online) |
| Volume:
|
56 |
| Issue:
|
5 |
| Year:
|
2020 |
| Pages:
|
875-885 |
| Summary lang:
|
English |
| . |
| Category:
|
math |
| . |
| Summary:
|
The problem to maximize the information divergence from an exponential family is generalized to the setting of Bregman divergences and suitably defined Bregman families. (English) |
| Keyword:
|
Bregman divergence |
| Keyword:
|
relative entropy |
| Keyword:
|
exponential family |
| Keyword:
|
optimization |
| MSC:
|
52A41 |
| MSC:
|
62B05 |
| MSC:
|
62E15 |
| MSC:
|
62E17 |
| MSC:
|
94A17 |
| idMR:
|
MR4187777 |
| DOI:
|
10.14736/kyb-2020-5-0875 |
| . |
| Date available:
|
2020-12-16T15:57:32Z |
| Last updated:
|
2021-02-23 |
| Stable URL:
|
http://hdl.handle.net/10338.dmlcz/148488 |
| . |
| Reference:
|
[1] Ay, N.: An information-geometric approach to a theory of pragmatic structuring..Ann. Probab. 30 (2002), 416-436. Zbl 1010.62007, MR 1894113, 10.1214/aop/1020107773 |
| Reference:
|
[2] Ay, N., Knauf, A.: Maximizing multi-information..Kybernetika 42 (2006), 517-538. Zbl 1249.82011, MR 2283503 |
| Reference:
|
[3] Barndorff-Nielsen, O.: Information and Exponential Families in Statistical Theory..Wiley, 1978. Zbl 1288.62007, MR 0489333 |
| Reference:
|
[4] Csiszár, I., Matúš, F.: Closures of exponential families..Ann. Probab. 33 (2005), 582-600. Zbl 1068.60008, MR 2123202, 10.1214/009117904000000766 |
| Reference:
|
[5] Csiszár, I., Matúš, F.: Generalized maximum likelihood extimates for exponential families..Probab. Theory Related Fields 141 (2008), 213-246. MR 2372970, 10.1007/s00440-007-0084-z |
| Reference:
|
[6] Geiger, D., Meek, C., Sturmfels, B.: On the toric algebra of graphical models..Ann. Statist. 34 (2006), 1463-1492. Zbl 1104.60007, MR 2278364, 10.1214/009053606000000263 |
| Reference:
|
[7] Matúš, F.: Maximization of information divergences from binary i.i.d. sequences..In: Proc. IPMU 2 (2004), 1303-1306. |
| Reference:
|
[8] Matúš, F.: Optimality conditions for maximizers of the information divergence from an exponential family..Kybernetika 43 (2007), 731-746. MR 2376334 |
| Reference:
|
[9] Matúš, F.: Divergence from factorizable distributions and matroid representations by partitions..IEEE Trans. Inform. Theory 55 (2009), 5375-5381. MR 2597169, 10.1109/tit.2009.2032806 |
| Reference:
|
[10] Matúš, F., Ay, N.: On maximization of the information divergence from an exponential family..In: Proc. WUPES 2003, University of Economics, Prague 2003, pp. 199-204. |
| Reference:
|
[11] Matúš, F., Csiszár, I.: Generalized minimizers of convex integral functionals, Bregman distance, Pythagorean identities..Kybernetika 48 (2012), 637-689. MR 3013394 |
| Reference:
|
[12] Matúš, F., Rauh, J.: Maximization of the information divergence from an exponential family and criticality..In: Proc. IEEE International Symposium on Information Theory (ISIT2011), 2011. MR 2817016, 10.1109/isit.2011.6034269 |
| Reference:
|
[13] Montúfar, G., Rauh, J., Ay, N.: Expressive power and approximation errors of Restricted Boltzmann Machines..In: Proc. NIPS 2011. |
| Reference:
|
[14] Montúfar, G., Rauh, J., Ay, N.: Maximal information divergence from statistical models defined by neural networks..In: Proc. GSI, 2013, pp. 759-766. 10.1007/978-3-642-40020-9\_85 |
| Reference:
|
[15] Rauh, J.: Finding the maximizers of the information divergence from an exponential family..IEEE Trans. Inform. Theory 57 (2011), 3236-3247. MR 2817016, 10.1109/tit.2011.2136230 |
| Reference:
|
[16] Rauh, J.: Finding the Maximizers of the Information Divergence from an Exponential Family.Ph.D. Dissertation, Universität Leipzig, 2011. MR 2817016 |
| Reference:
|
[17] Rockafellar, R. T.: Convex Analysis..Princeton University Press, 1970. Zbl 1011.49013, MR 0274683, 10.1017/s0013091500010142 |
| Reference:
|
[18] Wang, N., Rauh, J., Massam, H.: Approximating faces of marginal polytopes in discrete hierarchical models..Ann. Statist. 47 (2019), 1203-1233. MR 3911110, 10.1214/18-aos1710 |
| . |
