Article
Full entry |
PDF
(0.4 MB)
Feedback
PDF
(0.4 MB)
Feedback
Keywords:
dispersion; entropy; kurtosis; partial ordering
dispersion; entropy; kurtosis; partial ordering
Summary:
Partial orderings and measures of information for continuous univariate random variables with special roles of Gaussian and uniform distributions are discussed. The information measures and measures of non-Gaussianity including the third and fourth cumulants are generally used as projection indices in the projection pursuit approach for the independent component analysis. The connections between information, non-Gaussianity and statistical independence in the context of independent component analysis is discussed in detail.
References:
[1] Barron, A. R.: Entropy and the central limit theorem. Ann. Probab. 14 (1986), 336-342. DOI 10.1214/aop/1176992632 | MR 0815975 | Zbl 0599.60024
[2] Bell, A. J., Sejnowski, T. J.: An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7 (1995), 1129-1159. DOI 10.1162/neco.1995.7.6.1129
[3] Bickel, P. J., Lehmann, E. L.: Descriptive statistics for nonparametric models. II: Location. Ann. Stat. 3 (1975), 1045-1069. DOI 10.1214/aos/1176343240 | MR 0395021 | Zbl 0321.62055
[4] Bickel, P. J., Lehmann, E. L.: Descriptive statistics for nonparametric models. III: Dispersion. Ann. Stat. 4 (1976), 1139-1158. DOI 10.1214/aos/1176343648 | MR 474620 | Zbl 0351.62031
[5] Bilodeau, M., Brenner, D.: Theory of Multivariate Statistics. Springer Texts in Statistics, Springer, New York (1999). DOI 10.1007/b97615 | MR 1705291 | Zbl 0930.62054
[6] Chernoff, H., Savage, I. R.: Asymptotic normality and efficiency of certain nonparametric test statistics. Ann. Math. Stat. 29 (1958), 972-994. DOI 10.1214/aoms/1177706436 | MR 0100322 | Zbl 0092.36501
[7] Cover, T. M., Thomas, J. A.: Elements of Information Theory. Wiley Series in Telecommunications, John Wiley & Sons, New York (1991). DOI 10.1002/0471200611 | MR 1122806 | Zbl 0762.94001
[8] Faivishevsky, L., Goldberger, J.: ICA based on a smooth estimation of the differential entropy. NIPS'08: Proceedings of the 21st International Conference on Neural Information Processing Systems Curran Associates, New York (2008), 433-440.
[9] J. L. Hodges, Jr., E. L. Lehmann: The efficiency of some nonparametric competitors of the $t$-test. Ann. Math. Stat. 27 (1956), 324-335. DOI 10.1214/aoms/1177728261 | MR 0079383 | Zbl 0075.29206
[10] Huber, P. J.: Projection pursuit. Ann. Stat. 13 (1985), 435-475. DOI 10.1214/aos/1176349519 | MR 0790553 | Zbl 0595.62059
[11] Hyvärinen, A.: New approximations of differential entropy for independent component analysis and projection pursuit. NIPS '97: Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems 10 MIT Press, Cambridge (1998), 273-279.
[12] Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley & Sons, New York (2001). DOI 10.1002/0471221317
[13] Jones, M. C., Sibson, R.: What is projection pursuit?. J. R. Stat. Soc., Ser. A 150 (1987), 1-36. DOI 10.2307/2981662 | MR 0887823 | Zbl 0632.62059
[14] Kim, K., Shevlyakov, G.: Why Gaussianity?. IEEE Signal Processing Magazine 25 (2008), 102-113. DOI 10.1109/MSP.2007.913700
[15] Kristiansson, E.: Decreasing Rearrangement and Lorentz $L(p,q)$ Spaces. Master Thesis. Department of Mathematics, Lulea University of Technology, Lulea (2002), Available at \brokenlink { http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.111.1244&rep=rep1{&type=pdf}}\kern0pt
[16] Kullback, S.: Information Theory and Statistics. Wiley Publication in Mathematical Statistics, John Wiley & Sons, New York (1959). MR 0103557 | Zbl 0088.10406
[17] Learned-Miller, E. G., III, J. W. Fisher: ICA using spacings estimates of entropy. J. Mach. Learn. Res. 4 (2004), 1271-1295. DOI 10.1162/jmlr.2003.4.7-8.1271 | MR 2103630 | Zbl 1061.62007
[18] Lindsay, B. G., Yao, W.: Fisher information matrix: A tool for dimension reduction, projection pursuit, independent component analysis, and more. Can. J. Stat. 40 (2012), 712-730. DOI 10.1002/cjs.11166 | MR 2998858 | Zbl 1349.62300
[19] Marshall, A. W., Olkin, I.: Inequalities: Theory of Majorization and Its Applications. Mathematics in Science and Engineering 143, Academic Press, New York (1979). DOI 10.1016/B978-0-12-473750-1.50001-1 | MR 0552278 | Zbl 0437.26007
[20] Miettinen, J., Nordhausen, K., Oja, H., Taskinen, S.: Deflation-based FastICA with adaptive choices of nonlinearities. IEEE Trans. Signal Process 62 (2014), 5716-5724. DOI 10.1109/TSP.2014.2356442 | MR 3273526 | Zbl 1394.94394
[21] Miettinen, J., Nordhausen, K., Oja, H., Taskinen, S., Virta, J.: The squared symmetric \hbox{FastICA} estimator. Signal Process. 131 (2017), 402-411. DOI 10.1016/j.sigpro.2016.08.028
[22] Miettinen, J., Taskinen, S., Nordhausen, K., Oja, H.: Fourth moments and independent component analysis. Stat. Sci. 30 (2015), 372-390. DOI 10.1214/15-STS520 | MR 3383886 | Zbl 1332.62196
[23] Nordhausen, K., Oja, H.: Independent component analysis: A statistical perspective. WIREs Comput. Stat. 10 (2018), Article ID e1440, 23 pages. DOI 10.1002/wics.1440 | MR 3850449
[24] Nordhausen, K., Oja, H.: Robust nonparametric inference. Annu. Rev. Stat. Appl. 5 (2018), 473-500. DOI 10.1146/annurev-statistics-031017-100247 | MR 3774756
[25] Oja, H.: On location, scale, skewness and kurtosis of univariate distributions. Scand. J. Stat., Theory Appl. 8 (1981), 154-168. MR 0633040 | Zbl 0525.62020
[26] Pečarić, J. E., Proschan, F., Tong, Y. L.: Convex Functions, Partial Orderings, and Statistical Applications. Mathematics in Science and Engineering 187, Academic Press, Boston (1992). DOI 10.1016/S0076-5392(13)60006-5 | MR 1162312 | Zbl 0749.26004
[27] Ryff, J. V.: On the representation of doubly stochastic operators. Pac. J. Math. 13 (1963), 1379-1386. DOI 10.2140/pjm.1963.13.1379 | MR 0163171 | Zbl 0125.08405
[28] Serfling, R.: Asymptotic relative efficiency in estimation. International Encyclopedia of Statistical Science Springer, Berlin (2011), 68-72. DOI 10.1007/978-3-642-04898-2_126
[29] Shannon, C. E.: A mathematical theory of communication. Bell Syst. Tech. J. 27 (1948), 379-423. DOI 10.1002/j.1538-7305.1948.tb01338.x | MR 0026286 | Zbl 1154.94303
[30] Staudte, R. G.: The shapes of things to come: Probability density quantiles. Statistics 51 (2017), 782-800. DOI 10.1080/02331888.2016.1277225 | MR 3669289 | Zbl 1387.62045
[31] Staudte, R. G., Xia, A.: Divergence from, and convergence to, uniformity of probability density quantiles. Entropy 20 (2018), Article ID 317, 10 pages. DOI 10.3390/e20050317 | MR 3862507
[32] Zwet, W. R. van: Convex Transformations of Random Variables. Mathematical Centre Tracts 7, Mathematisch Centrum, Amsterdam (1964). MR 0176511 | Zbl 0125.37102
[33] Vigneron, V., Jutten, C.: Fisher information in source separation problems. International Conference on Independent Component Analysis and Signal Separation Lecture Notes in Computer Science 3195, Springer, Berlin (2004), 168-176. DOI 10.1007/978-3-540-30110-3_22
[34] Virta, J.: On characterizations of the covariance matrix. Available at https://arxiv.org/abs/1810.01147 (2018), 11 pages.
[35] Virta, J., Nordhausen, K.: On the optimal non-linearities for Gaussian mixtures in \hbox{FastICA}. Latent Variable Analysis and Signal Separation Theoretical Computer Science and General Issues, Springer, Berlin (2017), 427-437. DOI 10.1007/978-3-319-53547-0_40
Search
Partner of
