| Title:
|
Properties of unique information (English) |
| Author:
|
Rauh, Johannes |
| Author:
|
Schünemann, Maik |
| Author:
|
Jost, Jürgen |
| Language:
|
English |
| Journal:
|
Kybernetika |
| ISSN:
|
0023-5954 (print) |
| ISSN:
|
1805-949X (online) |
| Volume:
|
57 |
| Issue:
|
3 |
| Year:
|
2021 |
| Pages:
|
383-403 |
| Summary lang:
|
English |
| . |
| Category:
|
math |
| . |
| Summary:
|
We study the unique information function $UI(T:X\setminus Y)$ defined by Bertschinger et al. within the framework of information decompositions. In particular, we study uniqueness and support of the solutions to the convex optimization problem underlying the definition of $UI$. We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of $T$, $X$ and $Y$. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of $UI(T:X\setminus Y)$, most notably when $T$ is binary. Optima in the relative interior of the optimization domain are solutions of linear equations if $T$ is binary. In the all binary case, we obtain a complete picture of where the optimizing probability distributions lie. (English) |
| Keyword:
|
information decomposition |
| Keyword:
|
unique information |
| MSC:
|
94A15 |
| MSC:
|
94A17 |
| idZBL:
|
Zbl 07442516 |
| idMR:
|
MR4299455 |
| DOI:
|
10.14736/kyb-2021-3-0383 |
| . |
| Date available:
|
2021-11-04T12:41:10Z |
| Last updated:
|
2022-02-24 |
| Stable URL:
|
http://hdl.handle.net/10338.dmlcz/149197 |
| . |
| Reference:
|
[1] Amari, Shun-ichi, Nagaoka, Hiroshi: Methods of Information Geometry..American Mathematical Society 2000. |
| Reference:
|
[2] Banerjee, P. Kr., Olbrich, E., Jost, J., Rauh, J.: Unique informations and deficiencies..In: Proc. Allerton, 2018. |
| Reference:
|
[3] Banerjee, P. Kr., Rauh, J., Montúfar, G.: Computing the unique information..In: Proc. IEEE ISIT 2018, pp. 141-145 |
| Reference:
|
[4] Bertschinger, N., Rauh, J., Olbrich, E., Jost, J., Ay, N.: Quantifying unique information..Entropy 16 (2014), 4, 2161-2183. |
| Reference:
|
[5] Diaconis, Persi, Sturmfels, Bernd: Algebraic algorithms for sampling from conditional distributions..Ann. Statist. 26 (1998), 363-397. |
| Reference:
|
[6] Fink, A.: The binomial ideal of the intersection axiom for conditional probabilities..J. Algebr. Combin. 33 (2011), 3, 455-463. |
| Reference:
|
[7] Finn, C., Lizier, J. T.: Pointwise partial information decomposition using the specificity and ambiguity lattices..Entropy 20 (2018), 4, 297. |
| Reference:
|
[8] Harder, M., Salge, Ch., Polani, D.: A bivariate measure of redundant information..Phys. Rev. E 87 (2013), 012130. |
| Reference:
|
[9] Ince, R.: Measuring multivariate redundant information with pointwise common change in surprisal..Entropy 19 (2017), 7, 318. |
| Reference:
|
[10] James, R., Emenheiser, J., Crutchfield, J.: Unique information via dependency constraints..J. Physics A 52 (2018), 1, 014002. |
| Reference:
|
[11] Makkeh, A., Theis, D. O., Vicente, R.: Bivariate partial information decomposition: The optimization perspective..Entropy 19 (2017), 10, 530. |
| Reference:
|
[12] Niu, X., Quinn, Ch.: A measure of synergy, redundancy, and unique information using information geometry..In: Proc. IEEE ISIT 2019. |
| Reference:
|
[13] Rauh, J., Banerjee, P. Kr., Olbrich, E., Jost, J.: Unique information and secret key decompositions..In: 2019 IEEE International Symposium on Information Theory (ISIT), pp. 3042-3046. |
| Reference:
|
[14] Rauh, J., Ay, N.: Robustness, canalyzing functions and systems design..Theory Biosciences 133 (2014), 2, 63-78. |
| Reference:
|
[15] Rauh, J., Bertschinger, N., Olbrich, E., Jost, J.: Reconsidering unique information: Towards a multivariate information decomposition..In: Proc. IEEE ISIT 2014, pp. 2232-2236. |
| Reference:
|
[16] Smith, N. A., R, Tromble, W.: Sampling Uniformly from the Unit Simplex..Technical Report 29, Johns Hopkins University 29, 2004. |
| Reference:
|
[17] Williams, P., Beer, R.: Nonnegative decomposition of multivariate information..arXiv:1004.2515v1. |
| . |
