Shrinkage estimators' properties in regression models using L_1 penalized norm

Document Type : Original Article

Author

Department of Statistics, University of Qom, Qom, Iran

Abstract

In this paper, we first introduce a two-level hierarchical model with a linear structure.  We use the moment, maximum likelihood, and SURE estimators to obtain the regression coefficients shrinkage estimators. Since, regression models have vast applications in high-dimensional datasets, using sparsity assumption, we discuss the asymptotic properties of the regression estimators under L_2 error norm and L_1 penalty norm. 

Keywords

Main Subjects

References

[1] Baranchik, A.J. (1970). A Family of Minimax Estimators of the Mean of a Multivariate Normal Distribution. Ann. Math. Statist, 41, 642–645. DOI: https://doi.org/10.1214/aoms/1177697104.
[2] Berger, J. (1976). Admissible Minimax Estimation of a Multivariate Normal Mean With Arbitrary Quadratic Loss. The Annals of Statistics, 4, 223–226. DOI: https://doi.org/10.1214/aos/1176343356.
[3] Berger, J., & Strawderman, W.E. (1996). Choice of Hierarchical Priors:Admissibility in Estimation of Normal Means. The Annals of Statistics, 24, 931–951. DOI: https://doi.org/10.1214/aos/1032526950. 
[4] Brown, L.D. (1971). Admissible Estimators, Recurrent Diffusions, and Insoluble Boundary Value Problems. Ann. Math. Statist, 42, 855–903. DOI: https://doi.org/10.1214/aoms/1177693318.
[5] Buhlmann, P. (2013). Statistical significance in high-dimensional linear models. Bernoulli, 19, 1212–1242. DOI: https://doi.org/10.3150/12-BEJSP11.
[6] Buhlmann, P., & van de Geer, S. (2011). Statistics for high-dimensional data. Springer-Verlag. DOI: https://doi.org/10.1007/978-3-642-20192-9.
[7] Cai, T.T., & Zijian, Guo. (2016). Accuracy assessment for high-dimensional linear regression. arXiv preprint arXiv: 1603.03474. DOI: https://doi.org/10.48550/arXiv.1603.03474.
[8] Efron, B., & Morris, C. (1973). Stein’s Estimation Rule and Its Competitors: An Empirical Bayes Approach. J. Amer. Statist. Assoc, 68, 117–130. DOI: https://doi.org/10.2307/2284155.
[9] Ghoreishi, S.K., & Meshkani, M.R. (2014). On SURE estimators in hierarchical models assuming heteroscedasticity for both levels of a two-level normal hierarchical model. J. of Multivariate Analysis, 132, 129–137. DOI: https://doi.org/10.1016/j.jmva.2014.08.001.
[10] James, W., & Stein, C.M. (1961). Estimation With Quadratic Loss. Proceedings of the 4th Berkeley Symposium on Probability and Statistics, 1, 367–379. DOI: https://doi.org/10.1007/978-1-4612-0919-5_30.
[11] Morris, C. (1983). Parametric Empirical Bayes Inference: Theory and Applications. J. Amer. Statist. Assoc, 78, 47–55. DOI: https://doi.org/10.2307/2287098.
[12] Shanita, V., & Ghoreishi, S.K. (2020). Empirical Estimation for Sparse Double-Heteroscedastic Hierarchical Normal Models. Journal of Statistical Theory and Applications, 19, 148–161. DOI: https://doi.org/10.2991/jsta.d.200422.001.
[13] Stein, C.M. (1962). Confidence Sets for the Mean of a Multivariate Normal Distribution (with discussion). J. Roy. Statist. Soc. Ser. B, 24, 265–296. DOI: https://doi.org/10.1111/j.2517-6161.1962.tb00458.x.
[14] Xie, X., Kou, S.C., & Brown, L.D. (2012). SURE Estimates for a Heteroscedastic Hierarchical Model. J. Amer. Statist. Assoc, 107, 1465–1479. DOI: https://doi.org/10.1080/01621459.2012.728154.
[15] Xie, X., Kou, S.C., & Brown, L.D. (2016). Optimal shrinkage estimation of mean parameters in family of distributions with quadratic variance. The Annals of Statistics, 44, 564–597. DOI: https://doi.org/10.1214/15-AOS1377. 
Measure Algebras and Applications
Volume 1, Issue 1 - Serial Number 1
September 2023
Pages 97-106

Files

History

  • Receive Date: 15 May 2023
  • Revise Date: 07 August 2023
  • Accept Date: 13 August 2023

Share

How to cite

Statistics