On Model Selection Criterion for Finite Gaussian Mixture Models
Keywords:
Finite Mixture Models (FMM), Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Integrated Completed Likelihood (ICL), Bootstrap Likelihood Ratio Test (BLRT)Abstract
This paper delves into the realm of model selection criteria for Finite Mixture Models (FMM), focusing on key evaluation methods such as the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Integrated Completed Likelihood (ICL), and the Bootstrap Likelihood Ratio Test (BLRT). These criteria aid in balancing model fit and complexity, guiding researchers in choosing the most appropriate FMM for analyzing simulated and real datasets. Through extensive simulation studies, the paper meticulously evaluates and contrasts the performance of these criteria under various parameter settings and sample sizes, offering valuable insights for advancements in statistical modeling. The study underscores the importance of selecting the right criterion tailored to the dataset characteristics and research objectives. It highlights the impact of sample size on model selection, noting AIC's tendency to favor complexity and potential overfitting, while BIC and ICL excel in handling sample size variations by penalizing complexity effectively. The utilization of BLRT for comparing models with different complexities aids in identifying the optimal model configuration. Statistical analyses, including p-value assessments and visual aids like scatter plots and density functions, enhance the understanding of model performance and complexity. Overall, the paper emphasizes the significance of informed model selection decisions, ensuring a robust and accurate representation of underlying models in regression analysis.
References
Abdalla, A., & Michael, S. (2019). A finite mixture of regression models for a stratified sample. Journal of Statistical Computation and Simulation, 89(14), 2782-2800
Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. N. Petrov & F. Caski (Eds.), Second International Symposium on Information Theory (pp. 267-281). Akademiai Kiado
Baudry, J. P., Raftery, A. E., Celeux, G., Lo, K., & Gottardo, R. (2010). Combining mixture components for clustering. Journal of computational and graphical statistics, 19(2), 332-353.
Biernacki, C., Celeux, G., & Govaert, G. (2000). Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(7), 719-725.
Bozdogan, H. (1987). Model selection and Akaike’s Information Criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52(3), 345–370.
Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference: understanding AIC and BIC in model selection. Sociological methods & research, 33(2), 261-304.
Celeux, G. (2015). On the different ways to compute the integrated completed likelihood criterion.
Davison, A. and Hinkley, D. (1997) Bootstrap Methods and Their Applications. Cambridge University Press.
De Veaux, R. D. (1989). A review of mixture models. Journal of the American Statistical Association, 84(406), 446-454.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the royal statistical society: series B (methodological), 39(1), 1-22.
Fraley, C., & Raftery, A. E. (1998). How many clusters? Which clustering method? Answers via model-based cluster analysis. The Computer Journal, 41(8), 578-588.
Frühwirth-Schnatter, S. (2006). Finite mixture and Markov switching models. Springer.
Grimm, K. J., An, Y., McArdle, J. J., Zonderman, A. B., & Resnick, S. M. (2017). Recent changes leading to subsequent changes: Extensions of multivariate latent difference score models. Structural Equation Modeling: A Multidisciplinary Journal, 24(3), 321-337.
Jones, M. C., & McLachlan, G. J. (1992). Estimation in a family of linear regression models with mixtures of regressions. Journal of the American Statistical Association, 87(420), 958-967.
Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the american statistical association, 90(430), 773-795.
Lindsay, B. G. (1995). Mixture models: Theory, geometry, and applications. IMS Lecture Notes-Monograph Series, 34, 1-70.
McLachlan G.J. (1987) On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture. Applied Statistics, 36, 318-324.
McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York, NY: Wiley.
McLachlan, G. J., & Rathnayake, S. (2019). Finite mixture models. New York, NY: Wiley.
McLachlan, G., & Basford, K. E. (1988). Mixture models: Inference and applications to clustering. New York, NY: Marcel Dekker.
McLachlan, G., & Peel, D. (2000). Finite mixture models. New York, NY: Wiley.
McLachlan, G., Lee, S. X., & Rathnayake, S. (2019). Finite mixture models. New York, NY: Wiley.
McLachlan, G.J. and Peel, D. (2000) Finite Mixture Models. Wiley.
McLachlan, G.J. and Rathnayake, S. (2014) On the number of components in a Gaussian mixture model. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery,4(5), pp. 341-355.
McNicholas, P. D. (2016). Mixture model-based classification. New York, NY: Chapman and Hall/CRC.
Melnykov, V., Maitra, R., Ram, S., & Fu, M. (2015). Handbook of mixture analysis. New York, NY: CRC Press.
Pearson, K. (1894). Contributions to the mathematical theory of evolution. II. Skew variation in homogeneous material. Philosophical Transactions of the Royal Society of London. Series A, 185, 71-110.
Peel, D., & McLachlan, G. J. (2000). Robust mixture modelling using the t distribution. Statistics and computing, 10, 339-348.
Quandt, R. E., & Ramsey, J. B. (1978). Estimating mixtures of normal distributions and switching regressions. Journal of the American Statistical Association, 73(364), 730-738.
Rabe-Hesketh, S., & Skrondal, A. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. New York, NY: CRC Press.
Schlattmann, P. (2009). Medical applications of finite mixture models. New York, NY: Springer.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461-464.
Wedel, M., & Kamakura, W. A. (2012). Market segmentation: Conceptual and methodological foundations. New York, NY: Springer.
Wu, C. F. J. (1983). On the convergence properties of the EM algorithm. The Annals of Statistics, 11(1), 95-103.
Zucchini, W. (2000). An Introduction to Model Selection. Journal of Mathematical Psychology, 44, 41–61.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 International Journal of Sciences: Basic and Applied Research (IJSBAR)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who submit papers with this journal agree to the following terms.
