Article
Full entry |
PDF
(0.5 MB)
Feedback
PDF
(0.5 MB)
Feedback
Keywords:
probabilistic merging; information geometry; Kullback–Leibler divergence; Rényi entropy
probabilistic merging; information geometry; Kullback–Leibler divergence; Rényi entropy
Summary:
In this paper we present a result that relates merging of closed convex sets of discrete probability functions respectively by the squared Euclidean distance and the Kullback-Leibler divergence, using an inspiration from the Rényi entropy. While selecting the probability function with the highest Shannon entropy appears to be a convincingly justified way of representing a closed convex set of probability functions, the discussion on how to represent several closed convex sets of probability functions is still ongoing. The presented result provides a perspective on this discussion. Furthermore, for those who prefer the standard minimisation based on the squared Euclidean distance, it provides a connection to a probabilistic merging operator based on the Kullback-Leibler divergence, which is closely connected to the Shannon entropy.
References:
[1] Adamčík, M.: The information geometry of Bregman divergences and some applications in multi-expert reasoning. Entropy 16 (2014), 6338-6381. DOI 10.3390/e16126338 | MR 3299534
[2] Adamčík, M.: Collective Reasoning under Uncertainty and Inconsistency.
[3] Adamčík, M.: On the applicability of the ‘number of possible states’ argument in multi-expert reasoning. J. Appl. Logic 19 (2016), 20-49. DOI 10.1016/j.jal.2016.10.001 | MR 3573263
[4] Adamčík, M.: A logician's approach to meta-analysis with unexplained heterogeneity. J. Biomed. Inform. 71 (2017), 110-129. DOI 10.1016/j.jbi.2017.05.017
[5] Adamčík, M., Wilmers, G. M.: Probabilistic merging operators. Logique Analyse 228 (2014), 563-590. DOI 10.2143/LEA.228.0.3078175 | MR 3379218
[6] Amari, S., Cichocki, A.: Families of Alpha- Beta- and Gamma- divergences: Flexible and robust measures of similarities. Entropy 12 (2010), 1532-1568. DOI 10.3390/e12061532 | MR 2659408
[7] Basu, A., Harris, I. R., Hjort, N., Jones, M.: Robust and efficient estimation by minimising a density power divergence. Biometrika 85 (1998), 549-559. DOI 10.1093/biomet/85.3.549 | MR 1665873
[8] Bregman, L. M.: The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming. USSR Comput. Mathematics Math. Physics 1 (1967), 200-217. DOI 10.1016/0041-5553(67)90040-7 | MR 0215617
[9] Hawes, P.: Investigation of Properties of Some Inference Processes.
[10] Jaynes, E. T.: Where do we stand on maximum entropy?. In: The Maximum Entropy Formalism (R. D. Levine, M. Tribus, eds.), M.I.T. Press, 1979, pp. 15-118. MR 0521743
[11] Kern-Isberner, G., Rödder, W.: Belief revision and information fusion on optimum entropy. Int. J. Intell. Systems 19 (2004), 837-857. DOI 10.1002/int.20027 | Zbl 1101.68944
[12] Osherson, D., Vardi, M.: Aggregating disparate estimates of chance. Games Econom. Behavior 56 (2006), 148-173. DOI 10.1016/j.geb.2006.04.001 | MR 2235941 | Zbl 1127.62129
[13] Paris, J. B.: The Uncertain Reasoner Companion. Cambridge University Press, Cambridge 1994. MR 1314199
[14] Paris, J. B., Vencovská, A.: On the applicability of maximum entropy to inexact reasoning. Int. J. Approx. Reason. 3 (1989), 1-34. DOI 10.1016/0888-613x(89)90012-1 | MR 0975613 | Zbl 0665.68079
[15] Paris, J. B., Vencovská, A.: A note on the inevitability of maximum entropy. Int. J. Approx. Reason. 4 (1990), 183-224. DOI 10.1016/0888-613x(90)90020-3 | MR 1051032
[16] Predd, J. B., Osherson, D. N., Kulkarni, S. R, Poor, H. V.: Aggregating probabilistic forecasts from incoherent and abstaining experts. Decision Analysis 5 (2008), 177-189. DOI 10.1287/deca.1080.0119
[17] Rényi, A.: On measures of entropy and information. In: Proc. Fourth Berkeley Symposium on Mathematics, Statistics and Probability 1 (1961), 547-561. MR 0132570 | Zbl 0106.33001
[18] Shannon, C. E.: A mathematical theory of communication. Bell System Techn. J. 27 (1948), 379-423, 623-656. DOI 10.1002/j.1538-7305.1948.tb00917.x | MR 0026286 | Zbl 1154.94303
[19] Wilmers, G. M.: A foundational approach to generalising the maximum entropy inference process to the multi-agent context. Entropy 17 (2015), 594-645. DOI 10.3390/e17020594 | MR 3315866
Search
Partner of
