| Title:
|
The LASSO estimator: Distributional properties (English) |
| Author:
|
Jagannath, Rakshith |
| Author:
|
Upadhye, Neelesh S. |
| Language:
|
English |
| Journal:
|
Kybernetika |
| ISSN:
|
0023-5954 (print) |
| ISSN:
|
1805-949X (online) |
| Volume:
|
54 |
| Issue:
|
4 |
| Year:
|
2018 |
| Pages:
|
778-797 |
| Summary lang:
|
English |
| . |
| Category:
|
math |
| . |
| Summary:
|
The least absolute shrinkage and selection operator (LASSO) is a popular technique for simultaneous estimation and model selection. There have been a lot of studies on the large sample asymptotic distributional properties of the LASSO estimator, but it is also well-known that the asymptotic results can give a wrong picture of the LASSO estimator's actual finite-sample behaviour. The finite sample distribution of the LASSO estimator has been previously studied for the special case of orthogonal models. The aim in this work is to generalize the finite sample distribution properties of LASSO estimator for a real and linear measurement model in Gaussian noise. In this work, we derive an expression for the finite sample characteristic function of the LASSO estimator, we then use the Fourier slice theorem to obtain an approximate expression for the marginal probability density functions of the one-dimensional components of a linear transformation of the LASSO estimator. (English) |
| Keyword:
|
linear regression |
| Keyword:
|
LASSO |
| Keyword:
|
characteristic function |
| Keyword:
|
finite sample probability distribution function |
| Keyword:
|
Fourier-Slice theorem |
| Keyword:
|
Cramer–Wold theorem |
| MSC:
|
60E05 |
| MSC:
|
62E15 |
| MSC:
|
62G05 |
| MSC:
|
62J05 |
| idZBL:
|
Zbl 06987034 |
| idMR:
|
MR3863256 |
| DOI:
|
10.14736/kyb-2018-4-0778 |
| . |
| Date available:
|
2018-10-30T14:51:44Z |
| Last updated:
|
2023-10-03 |
| Stable URL:
|
http://hdl.handle.net/10338.dmlcz/147424 |
| . |
| Reference:
|
[1] Austin, C. D., Moses, R., Ash, J., Ertin, E.: On the relation between sparse reconstruction and parameter estimation with model order selection..IEEE J. Selected Topics Signal Process. 4 (2010), 3, 560-570. 10.1109/jstsp.2009.2038313 |
| Reference:
|
[2] Babacan, S., Molina, R., Katsaggelos, A.: Bayesian compressive sensing using laplace priors..IEEE Trans. Image Process. 19 (2010), 1, 53-63. MR 2729957, 10.1109/tip.2009.2032894 |
| Reference:
|
[3] Baraniuk, R., Candes, E., Nowak, R., R., M., Vetterli: Compressive sampling..IEEE Signal Processing Magazine 25 (2008), 2, 12-13. 10.1109/msp.2008.915557 |
| Reference:
|
[4] Ben-Tal, A., Nemirovskiaei, A. S.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications..Society for Industrial and Applied Mathematics, Philadelphia 2001. MR 1857264, 10.1137/1.9780898718829 |
| Reference:
|
[5] Boufounos, P., Duarte, M. F., Baraniuk, R. G.: Sparse signal reconstruction from noisy compressive measurements using cross validation..In: IEEE/SP 14th Workshop on Statistical Signal Processing, 2007, pp. 299-303. MR 2883008, 10.1109/ssp.2007.4301267 |
| Reference:
|
[6] Candes, E.: The restricted isometry property and its implications for compressed sensing..Comptes Rendus Mathematique 346 (2008), 9-10, 589-592. MR 2412803, 10.1016/j.crma.2008.03.014 |
| Reference:
|
[7] Chen, S. S., Donoho, D. L., Saunders, M. A.: Atomic decomposition by basis pursuit..SIAM Rev. 43 (2001), 1, 129-159. MR 1854649, 10.1137/s003614450037906x |
| Reference:
|
[8] Cuesta-Albertos, J. A., Fraiman, R., Ransford, T.: A sharp form of the Cramér-wold theorem..J. Theoret. Probab. 20 (2007), 2, 201-209. MR 2324526, 10.1007/s10959-007-0060-7 |
| Reference:
|
[9] Donoho, D.: Compressed sensing..IEEE Trans. Inform. Theory 52 (2006), 4, 1289-1306. MR 2241189, 10.1109/tit.2006.871582 |
| Reference:
|
[10] Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression..Ann. Statist. 32 (2004), 407-499. MR 2060166, 10.1214/009053604000000067 |
| Reference:
|
[11] Eldar, Y.: Generalized sure for exponential families: Applications to regularization..IEEE Trans. Signal Process. 57 (2009), 2, 471-481. MR 2603376, 10.1109/tsp.2008.2008212 |
| Reference:
|
[12] Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties..J. Amer. Statist. Assoc. 96 (2001), 456, 1348-1360. MR 1946581, 10.1198/016214501753382273 |
| Reference:
|
[13] Grant, M., Boyd, S.: {CVX}: Matlab software for disciplined convex programming, version 2.1.. |
| Reference:
|
[14] Kabaila, P.: The effect of model selection on confidence regions and prediction regions..Econometr. Theory 11 (1995), 537-549. MR 1349934, 10.1017/s0266466600009403 |
| Reference:
|
[15] Kay, S. M.: Fundamentals of Statistical Signal Processing: Estimation Theory..Prentice-Hall, Inc., Upper Saddle River, NJ 1993. |
| Reference:
|
[16] Knight, K., Fu, W.: Asymptotics for lasso-type estimators..Ann. Statist. 28 (2000), 5, 1356-1378. MR 1805787, 10.1214/aos/1015957397 |
| Reference:
|
[17] Krim, H., Viberg, M.: Two decades of array signal processing research: the parametric approach..IEEE Signal Processing Magazine 13 (1996), 4, 67-94. 10.1109/79.526899 |
| Reference:
|
[18] Leeb, H., Pötscher, B. M.: Model selection and inference: Facts and fiction..Econometr. Theory 21 (2005), 21-59. MR 2153856, 10.1017/s0266466605050036 |
| Reference:
|
[19] Lockhart, R., Taylor, J., Tibshirani, R. J., Tibshirani, R.: A significance test for the lasso..Ann. Statist. 42 (2014), 2, 413-468. MR 3210970, 10.1214/13-aos1175 |
| Reference:
|
[20] Lopes, M. E.: Estimating unknown sparsity in compressed sensing..CoRR 2012, abs/1204.4227. |
| Reference:
|
[21] Ng, R.: Fourier slice photography..ACM Trans. Graph. 24 (2005), 3, 735-744. 10.1145/1073204.1073256 |
| Reference:
|
[22] Panahi, A., Viberg, M.: Fast candidate points selection in the lasso path..IEEE Signal Process. Lett. 19 (2012), 2, 79-82. 10.1109/lsp.2011.2179534 |
| Reference:
|
[23] Pötscher, B. M., Leeb, H.: On the distribution of penalized maximum likelihood estimators: The lasso, scad, and thresholding..J. Multivar. Anal. 100 (2009), 9, 2065-2082. MR 2543087, 10.1016/j.jmva.2009.06.010 |
| Reference:
|
[24] Tibshirani, R.: Regression shrinkage and selection via the lasso..J. Roy. Statist. Soc., Ser. B 58 (1994), 267-288. MR 1379242 |
| Reference:
|
[25] Zou, H.: The adaptive lasso and its oracle properties..J. Amer. Statist. Assoc. 101 (2006), 476, 1418-1429. MR 2279469, 10.1198/016214506000000735 |
| . |
