A Comparative Study on Classical Test Theory and Rasch Model in Comprehensive Mental Ability Test (CMAT): A Pilot Test

Josephine P. Manapsal

Abstract


This study is a comparative analysis of Classical Test Theory (CTT) and the Rasch Model in the pilot testing of Comprehensive Mental Ability Test (CMAT). It investigated whether bad items in CTT would also be bad or unfit items when the Rasch model is applied. Thirty-three Masters level students in one of the state universities in Manila were the participants of the study. Sixty minutes were given for them to answer the 60 items of CMAT. Using CTT, the reliability test result showed that the computed KR#20 is .76 while using Rasch, the computed person ability reliability is .73.Out of 60 items, 5 items were found to be bad items using CTT and 17 items were found to be misfit using Rasch Analysis. This result showed that using the Rasch Model, it becomes stricter than CTT. Also, there are items discarded in CTT but not in the Rasch model, and there are items found to be unfit in the Rasch model but they are good items in the CTT. Thus, each particular model has their own specific parameter, whether which one will offer a better outcome for test construction and development, the answer will rely on the parameter of the tests, orientation of the test developer and the purpose of the test.


Keywords


Item difficulty; item discrimination; IRT; item calibration; misfit items; unidimensionality; overfit; infit; logits.

Full Text:

PDF

References


C. Cantell,. Item response theory: Understanding the one-parameter Rasch Model. A paper presented at the annual meeting of the Southwest Educational Research Association Austin TX, 1997.

R.J. Cohen. Exercises in psychological testing and assessment: An Introduction to tests and measurements 6th ed., New York: McGraw-Hill International. 2005.

R.J. Cohen, & M.E. Swerdlik. Psychological testing and assessment: An Introduction to tests and measurements 6th ed., New York: McGraw-Hill International. 2007.

S.E. Embretson & S.P. Reise. Item response theory for psychologist. London: Lawrence Erlbaum Associates, Publishers, 2000.

F. Xitao. Item response theory and classical test theory: an empirical comparison of their item/person statistics. Educational and Psychological Measurement Sage Publication, vol. 58 n3 pp. 357 (25) June 1998.

J.E. Gustafsson. An introduction to Rasch measurement model. Paper presented at the Nordic Researchers Course Rasch Models in the social and behavioral sciences. Princeton, NJ, 1990.

Henson, R.. Understanding the one-parameter Rash Model of item response model of the item response theory. A paper presented at the annual meeting of the Southwest Educational Research Association San Antonio,TX. 1999

Kubiszyn, T. & Borick, (2004) Educational testing and measurement Classroom Application and Practice 7thed. New York: John Wiley& Sons, Inc. ,2004.

Linacre J.M. True-score reliability or Rasch statistical validity? Rasch Measurement Transaction 9:4 p. 455-6, 1996.

Lord, F. M.. Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum. 1980

Magno, C. & Hai, CY. The Application of a One- parameter IRT Model on A Test of Mathematical Problem Solving.

Reeve, B. B. Item response theory modeling in health outcomes measurement. Expert Review of Pharmacoeconomics and Outcomes Research, 3 (2), 131-145. 2003

B. B. Reeve, & P. Fayers, P. .Applying item response theory modeling for evaluating questionnaire item and scale properties. In P. Fayers and R. D. Hays (Eds.), Assessing Quality of Life in Clinical Trials: Methods of Practice. 2ndEdition. Oxford University Press. 2005. pp.55-73.

B.B. Reeve, R.D. Hays, J.B. Bjorner, K.F. Cook, P.K. Crane, J.A. Teresi, D. Thissen, D. A.Revicki, D.J. Weiss, R.K. Hambleton, H. Liu, R. Gershon, S.P. Reise, J.S. Lai, D. Cella,. Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care.45(5) S22-S31. 2007.

Wiberg, M. Classical Test theory vs. Item Response theory. EM No.50, 2004.

Wright, B. D. Sample-free test calibration and person measurement. In Proceedings of the 1967 Invitational Conference on Testing Problems ,Princeton, N.J.: Educational Testing Service.1967.

Wright, B. D. & N. Panchepakesan,. A procedure for sample free item analysis. Educational and Psychological Measurement, 1969. 29, 23-37.

B. D. Wright & R. J. Mead. Calfit: Sample-free item calibration with a Rasch measurement model. Research Memorandum, No. 18. Statistical Laboratory, Department of Education, University of Chicago, 1975

B. D. Wright & G. A. Douglas. Better procedures for sample-free item analysis. Research Memorandum, No. 20 .Statistical Laboratory, Department of Education, University of Chicago. 1975

B. D. Wright & M.H. Stone. Best Test Designs and Self Tailored Testing. Research Memorandum , No. 19. Statistical Laboratory, Department of Education, University of Chicago., 1975

B. D. Wright & M.H. Stone. Best Test Designs. Chicago: Mesa Press, 1979.

T.G. Bond, & C.M. Fox. Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Rasch Measurement Transactions, 2001, 15:1 p.790.

B.D.Wright & JM Linacre. Reasonable mean-square fit values. Rasch Measurement Transactions, 1994, 8:3 p.370.


Refbacks

  • There are currently no refbacks.


 

 
  
 

 

  


About IJSBAR | Privacy PolicyTerms & Conditions | Contact Us | DisclaimerFAQs 

IJSBAR is published by (GSSRR).