| Title:
|
Maximizing multi–information (English) |
| Author:
|
Ay, Nihat |
| Author:
|
Knauf, Andreas |
| Language:
|
English |
| Journal:
|
Kybernetika |
| ISSN:
|
0023-5954 |
| Volume:
|
42 |
| Issue:
|
5 |
| Year:
|
2006 |
| Pages:
|
517-538 |
| Summary lang:
|
English |
| . |
| Category:
|
math |
| . |
| Summary:
|
Stochastic interdependence of a probability distribution on a product space is measured by its Kullback–Leibler distance from the exponential family of product distributions (called multi-information). Here we investigate low-dimensional exponential families that contain the maximizers of stochastic interdependence in their closure. Based on a detailed description of the structure of probability distributions with globally maximal multi-information we obtain our main result: The exponential family of pure pair-interactions contains all global maximizers of the multi-information in its closure. (English) |
| Keyword:
|
multi-information |
| Keyword:
|
exponential family |
| Keyword:
|
relative entropy |
| Keyword:
|
pair- interaction |
| Keyword:
|
infomax principle |
| Keyword:
|
Boltzmann machine |
| Keyword:
|
neural networks |
| MSC:
|
60B10 |
| MSC:
|
82C32 |
| MSC:
|
92B20 |
| MSC:
|
94A15 |
| idZBL:
|
Zbl 1249.82011 |
| idMR:
|
MR2283503 |
| . |
| Date available:
|
2009-09-24T20:18:24Z |
| Last updated:
|
2015-03-29 |
| Stable URL:
|
http://hdl.handle.net/10338.dmlcz/135733 |
| . |
| Reference:
|
[1] Aarts E., Korst J.: Simulated Annealing and Boltzmann Machines.Wiley, New York 1989 Zbl 0674.90059, MR 0983115 |
| Reference:
|
[2] Ackley D. H., Hinton G. E., Sejnowski T. J.: A learning algorithm for Boltzmann machines.Cognitive Science 9 (1985), 147–169 10.1207/s15516709cog0901_7 |
| Reference:
|
[3] Aigner M.: Combinatorial Theory, Classics in Mathematics.Springer–Verlag, Berlin 1997 MR 1434477 |
| Reference:
|
[4] Amari S.: Information geometry on hierarchy of probability distributions.IEEE Trans. Inform. Theory 47 (2001), 1701–1711 Zbl 0997.94009, MR 1842511, 10.1109/18.930911 |
| Reference:
|
[5] Amari S., Kurata, K., Nagaoka H.: Information geometry of Boltzmann machines.IEEE Trans. Neural Networks 3 (1992), 2, 260–271 10.1109/72.125867 |
| Reference:
|
[6] Ay N.: An information-geometric approach to a theory of pragmatic structuring.Ann. Probab. 30 (2002), 416–436 Zbl 1010.62007, MR 1894113, 10.1214/aop/1020107773 |
| Reference:
|
[7] Ay N.: Locality of global stochastic interaction in directed acyclic networks.Neural Computation 14 (2002), 2959–2980 Zbl 1079.68582, 10.1162/089976602760805368 |
| Reference:
|
[8] Linsker R.: Self-organization in a perceptual network.IEEE Computer 21 (1988), 105–117 10.1109/2.36 |
| Reference:
|
[9] Matúš F., Ay N.: On maximization of the information divergence from an exponential family.In: Proc. WUPES’03 (J. Vejnarová, ed.), University of Economics, Prague 2003, pp. 199–204 |
| Reference:
|
[10] Shannon C. E.: A mathematical theory of communication.Bell System Tech. J. 27 (1948), 379–423, 623–656 Zbl 1154.94303, MR 0026286, 10.1002/j.1538-7305.1948.tb01338.x |
| Reference:
|
[11] Tononi G., Sporns, O., Edelman G. M.: A measure for brain complexity: Relating functional segregation and integration in the nervous system.Proc. Nat. Acad. Sci. U. S. A. 91 (1994), 5033–5037 10.1073/pnas.91.11.5033 |
| . |
