Document Type : Research Article
Authors
1 Allameh Tabataba'i University
2 Allameh, Tabatabai University
Abstract
The paper considers the problem of estimation of the parameters in nite mixture models.In this article, a new method is proposed for of estimation of the parameters in nite mixture models. Traditionally, the parameter estimation in nite mixture models is performed from a likelihood point of view by exploiting the expectation maximization (EM) method and the Least Square Principle. Ridge regression is an alternative to the ordinary least squares method when multicollinearity presents among the regressor variables in multiple linear regression analysis. Accordingly, we propose a new shrinkage ridge estimation approach. Based on this principle, we propose an iterative algorithm called RidgeIterative Weighted least Square (RIWLS) to estimate the parameters. Monte-Carlo simulation studies are conducted to appraise the performance of our method. The results show that the Proposed estimator perform better than the IWLS method.
Keywords
[2] Carroll, R. J., and P. Hall. 1988. Optimal rates of convergence for deconvolving a density. Journal of the American Statistical Association 83 (404):11846. :10.2307/2290153.
[3] DasGupta, A. 2008. Mixture Models and Nonparametric Deconvolution. In Asymptotic Theory of Statistics and Probability, 57191. New York: Springer.
[4] De Veaux, R. D. 1989. Mixtures of linear regressions. Computational Statistics & Data Analysis, 8(3), 227-45. :10.1016/0167-9473(89)90043-1.
[5] Dias, J. G., and M. Wedel. 2004. An empirical comparison of EM, SEM and MCMC performance for problematic Gaussian mixture likelihoods. Statistics and Computing, 14(4), 323-32.:10.1023/B:STCO.0000039481.32211.5a.
[6] Elmore, R., P. Hall, and A. Neeman. 2005. An application of classical invariant theory to identi ability in nonparametric mixtures. In Annales de l'institut Fourier 55 (1):128. :10.5802/aif.2087.
[7] Eskandari, F., and E. Ormoz. 2016. Finite Mixture of Generalized Semiparametric Models: Variable Selection via Penalized Estimation. Communications in Statistics-Simulation andComputation, 45(10), 3744-59.:10.1080/03610918.2014.953687.
[8] Fan, J. 1991. On the optimal rates of convergence for nonparametric deconvolution problems.The Annals of Statistics, 1257-72. :10.1214/aos/1176348248.
[10] Faria, S., and G. Soromenho. 2012. Comparison of EM and SEM algorithms in Poisson regression models: A simulation study. Communications in Statistics-Simulation and Computation, 41(4): 497509. :10.1080/03610918.2011.594534.
[11] Frank, I.E., and J.H Friedman. 1993. An Statistical View of Some Chemometrics Regression Tools. Technometrics, 35, 109-135. : 10.1080/00401706.1993.10485033.
[12] Hall, P., A. Neeman, R. Pakyari, and R. Elmore. 2005. Nonparametric inference in multivariate mixtures. Biometrika 92(3), 667-78. :10.1093/biomet/92.3.667.
[13] Hall, P., and X. H. Zhou. 2003. Nonparametric estimation of component distributions in a multivariate mixture. The Annals of Statistics 31 (1):20124. :10.1214/aos/1046294462.
[14] Hawkins, D. S., D. M. Allen, and A. J. Stromberg. 2001. Determining the number of components in mixtures of linear models. Computational Statistics & Data Analysis 38(1), 15-48.:10.1016/S0167-9473(01)00017-2.
[15] Hoerl, A. E, and Kennard R. W.1970. Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55-67. : 10.1080/00401706.1970.10488634.
[16] Jacobs, R. A., Jordan, M. I., Nowlan, S. J. and Hinton, G. E. 1991. Adaptive mixture of local experts. Neural Computation 3, 79-87. : 10.1162/neco.1991.3.1.79.
[17] Jiang, W. and Tanner, M. A. 1999. Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation. The Annals of Statistics 27, 987-1011. : 10.1214/aos/1018031265.
[18] Jones, P. N., and G. J. McLachlan. 1992. Fitting nite mixture models in a regression context. Australian Journal of Statistics 34 (2):23340. :10.1111/j.1467-842X.1992.tb01356.x.
[19] McDonald, G. C., and D.I. Galarneau. 1975. A monte carlo evaluation of ridgetype estimators. Journal of the American Statistical Association 70 (350):40716. :10.1080/01621459.1975.10479882.
[20] McLachlan, G. J. and Peel, D. (2000), Finite Mixture Models, New York: Wiley.
[21] Nguyen, X. 2013. Convergence of latent mixing measures in nite and in nite mixture models. The Annals of Statistics 41 (1):370400. :10.1214/12-AOS1065.
[22] Pearson, K. 1894. Contributions to the mathematical theory of evolution. Philosophical Transactions of the Royal Society of London. A 185:71110. :10.1098/rsta.1894.0003.
[23] Pollard, D. 1991. Asymptotics for Least Absolute Deviation Regression Estimators. Econometric Theory 18699. : 10.1017/S0266466600004394.
[24] Quandt, R. E., and J. B. Ramsey. 1978. Estimating mixtures of normal distributions and switching regressions. Journal of the American statistical Association 73 (364):7308.:10.2307/2286266.
[25] Rezazadeh, H., F. Eskandari, M. Bameni Moghadam and E. Ormoz. 2020. Variable selection in nite mixture of generalized estimating equations.Communications in Statistics - Simulation and Computation.:10.1080/03610918.2019.1711406.
[26] Rousseau, J., and K. Mengersen. 2011. Asymptotic behaviour of the posterior distribution in over tted mixture models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 (5):689710. :10.1111/j.1467-9868.2011.00781.x.
[27] Schepers, J. 2015. Improved random-starting method for the EM algorithm for nite mixtures of regressions. Behavior research methods 47 (1):13446.
[28] Searle, S. R. 1997. Linear models. Hoboken, New Jersey: John Wiley & Sons.
[29] Teicher, H. 1961. Identi ability of mixtures. The Annals of Mathematical Statistics 32 (1):2448.
[30] Teicher, H. 1963. Identi ability of nite mixtures. The Annals of Mathematical Statistics 34 (4): 12659.:10.1214/aoms/1177703862.
[31] Tibshirani, R. 1996. Regression shrinkage and selection via the Lasso. Journ al of the Royal Statistical Society, Series B58, 267-88. : 10.1111/j.2517-6161.1996.tb02080.x.
[32] Xu, L., N. Lin, B. Zhang, and N. Shi. 2012. A Finite mixture model for working correlation matrices in generalized estimating equations. Statistica Sinica 22 (2):75576.:10.5705/ss.2010.090.
[34] Zhang, C. H. 1990. Fourier methods for estimating mixing densities and distributions. The Annals of Statistics 18 (2):80631. :10.1214/aos/1176347627.
[35] Zhu, H. T., and H. Zhang. 2004. Hypothesis testing in mixture regression models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 66 (1):316. :10.1046/j.1369-7412.
)