Item Quality Analysis of Physics Concept Understanding Test with Rasch Model

Aris Kurniawan; Edi Istiyono; Sul Daeng Naba

doi:10.26737/jipf.v9i3.5692

Item Quality Analysis of Physics Concept Understanding Test with Rasch Model

Aris Kurniawan, Edi Istiyono, Sul Daeng Naba

Abstract

This study aims to determine the quality of physics concept understanding test items on the topic of electromagnetic waves based on estimation of item fit, item difficulty level estimation, and item reliability estimation. The research sample consisted of 298 eleventh-grade science students from several public high schools in Central Kalimantan province. The results of data analysis using the Rasch model with the aid of QUEST software showed that all items were fit or valid, with INFIT MNSQ values ranging from 0.77 to 1.33. The item reliability was 0.98, categorized as excellent, and the test participant reliability was 0.25, categorized as weak. The difficulty level of the test items was categorized into three: 5 items were difficult, 5 items were moderate, and 5 items were easy. Furthermore, based on the analysis using OUTFIT t values, one item was discarded and could not be used as a test instrument because it had an OUTFIT t value greater than 2.

Keywords

Concept Understanding; Rasch Model; Item Response Theory

Full Text:

PDF (English)

References

Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives: complete edition. Addison Wesley Longman, Inc..

Popham, W. J. (2008). Classroom assessment: What teachers need to know. Pearson.

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences, Third Edition. Psychology Press.

Pennington, M. C., & Black, S. L. (2010). Conceptual Understanding and Scientific Reasoning of High School Students in Physics. International Journal of Science Education, 32(5), 567–589.

Land, T. (2013). Conceptual understanding: The case of electricity and magnetism. European Journal of Science and Mathematics Education, 1(1), 13–28.

Bella, A. Z., Azizahwati, A., & Azhar, A. Pengembangan Instrumen Tes Pemahaman Konsep Materi Momentum dan Impuls. Jurnal Online Mahasiswa (JOM) Bidang Keguruan dan Ilmu Pendidikan, 8(1), 117-127.

Abdi, M. U., Mustafa, M., & Pada, A. U. T. (2021). Penerapan pendekatan STEM berbasis simulasi PhET untuk meningkatkan pemahaman konsep fisika peserta didik. JIPI (Jurnal IPA dan Pembelajaran IPA), 5(3), 209-218.

Riwanto, D., Azis, A., & Arafah, K. (2019). Analisis pemahaman konsep peserta didik dalam menyelesaikan soal-soal fisika kelas x mia sma negeri 3 soppeng. Jurnal Sains Dan Pendidikan Fisika, 15(2), 23-31.

Afifah, R. (2019). Analisis profil proses kognitif pemahaman konsep siswa. Jurnal Pendidikan Fisika, 7(2), 170-178.

Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory. Wadsworth Publishing.

Osterlind, S. J. (2006). Modern measurement: Theory, principles, and applications of mental appraisal. Prentice Hall.

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Sage Publications.

Baker, F. B., & Kim, S. H. (2004). Item response theory: Parameter estimation techniques. CRC press.

Rash, G. (1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research.

Embretson, S. E., & Reise, S. P. (2013). Item response theory. Psychology Press.

Salibašić Glamočić, D., Mešić, V., Neumann, K., Sušac, A., Boone, W. J., Aviani, I., ... & Grubelnik, V. (2021). Maintaining item banks with the Rasch model: An example from wave optics. Physical Review Physics Education Research, 17(1), 010105.

Purnami, W., Fauzi, A., & Naingalis, M. L. P. (2023, June). Computational thinking skills identification among students of physics education department using Rasch model analysis. In AIP Conference Proceedings (Vol. 2751, No. 1). AIP Publishing.

Planinic, M., Ivanjek, L., & Susac, A. (2010). Rasch model based analysis of the Force Concept Inventory. Physical Review Special Topics—Physics Education Research, 6(1), 010103.

Syadiah, A. N., & Hamdu, G. (2020). Analisis rasch untuk soal tes berpikir kritis pada pembelajaran STEM di sekolah dasar. Premiere Educandum: Jurnal Pendidikan Dasar Dan Pembelajaran, 10(2), 138-148.

Nisa, K., & Suprapto, N. (2023). Deteksi Bias Gender dan Domisili Menggunakan DIF (Differential Item Functioning): Analisis Instrumen Tes Keterampilan Pemecahan Masalah Terintegrasi Etnofisika. Inovasi Pendidikan Fisika, 12(1), 30-35.

Tarigan, E. F., Nilmarito, S., Islamiyah, K., Darmana, A., & Suyanti, R. D. (2022). Analisis Instrumen Tes Menggunakan Rasch Model dan Software SPSS 22.0. Jurnal Inovasi Pendidikan Kimia, 16(2), 92-96.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan rasch pada assessment pendidikan. Trim komunikata.

Maulana, S., Rusilowati, A., Nugroho, S. E., & Susilaningsih, E. (2023, June). Implementasi rasch model dalam pengembangan instrumen tes diagnostik. In Prosiding Seminar Nasional Pascasarjana (Vol. 6, No. 1, pp. 748-757).

Suryani, Y. E. (2018). Aplikasi rasch model dalam mengevaluasi Intelligenz Structure Test (IST). Psikohumaniora: Jurnal Penelitian Psikologi, 3(1), 73-100.

Islam, A. A., Gu, X., Crook, C., & Spector, J. M. (2020). Assessment of ICT in tertiary education applying structural equation modeling and Rasch model. Sage Open, 10(4), 2158244020975409.

Xiao, Y., Han, J., Koenig, K., Xiong, J., & Bao, L. (2018). Multilevel Rasch modeling of two-tier multiple choice test: A case study using Lawson’s classroom test of scientific reasoning. Physical Review Physics Education Research, 14(2), 020104.

Matore, M. E. E. M., Maat, S. M., Affandi, H. M., & Khairani, A. Z. (2018). Assessment of psychometric properties for Raven Advanced Progressive Matrices in measuring intellectual quotient (IQ) using Rasch model. Asian Journal of Scientific Research, 11(3), 393-400.

bin Khairani, A. Z., & bin Abd Razak, N. (2015). Modeling a multiple choice mathematics test with the Rasch model. Indian Journal of Science and Technology, 8(12), 1.

Ozarslan, M., & Çetin, G. (2018). Biology Students' Cognitive Structures about Basic Components of Living Organisms. Science Education International, 29(2).

Wulandari, T., Ramli, M., & Muzzazinah, M. (2022). Analisis butir soal dynamic assessment untuk mengukur pemahaman konsep klasifikasi tumbuhan pada mahasiswa. Jurnal Pendidikan Sains Indonesia (Indonesian Journal of Science Education), 10(1), 191-201.

Groundlund, R. L., & Linn, N. E. (1990). Measurement and Evaluation in Teaching. Prentice Hall College Div.

Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and Validating Test. Routledge Taylor & Francis Group.

Engeström, Y., Virkkunen, J., Helle, M., Pihlaja, J., & Poikela, R. (1996). The change laboratory as a tool for transforming work. Lifelong learning in Europe, 1(2), 10-17.

Mehrens, W. A., & Lehmann, I. J. (1991). Measurement and Evaluation in Education and Psychology, Fourth Ed. Wadsworth/Thomson Learning.

Handayani, Y., Rahmawati, R., & Widiasih, W. (2023). Using Rasch Model to Analyze Reliability and Validity of Concept Mastery Test on Electricity and Magnetism Topic. JIPF (Jurnal Ilmu Pendidikan Fisika), 8(2), 226-239.

Fiskawarni, T. H., Rahmawati, R., Widiasih, W., & Saad, R. (2024). The Design and Validation of the Four Tier Test Instrument for Energy Literacy Using the Rasch Model Analysis. JIPF (Jurnal Ilmu Pendidikan Fisika), 9(1), 74-87.

Hofer, S. I., Schumacher, R., & Rubin, H. (2017). The test of basic Mechanics Conceptual Understanding (bMCU): using Rasch analysis to develop and evaluate an efficient multiple choice test on Newton’s mechanics. International journal of STEM education, 4, 1-20.

Bozdağ, H. C., & Türkoğuz, S. (2021). A Rasch Model Analysis of Primary School Students’ Conceptual Understanding Levels of the Concept of Light. International Online Journal of Primary Education, 10(1), 160-179.

Andriani, R., Fadieny, N., & Permana, N. D. (2023). Development of Conceptual Understanding Student Tests to The Basic Physics Subject: a Rasch Model Analysis. Co-Catalyst: Journal of Science Education Research and Theories, 1(1), 43-54.

Retnawati, H. (2016). Analisis Kuantitatif Instrumen Penelitian (Panduan Peneliti, Mahasiswa, dan Psikometrian). Prama Publishing.

Setyawarno, D. (2017). Penggunaan AplikasiSoftware Iteman (Item and Test Analysis) untuk Analisis Butir Soal Pilihan Ganda Berdasarkan Teori Tes Klasik. Jurnal Ilmu Fisika Dan Pembelajarannya (JIFP), 1(1), 11-21.

Subali, B., & Suyata, P. (2011). Panduan analisis data pengukuran pendidikan untuk memperoleh bukti empirik kesahihan menggunakan program Quest. Lembaga Penelitian dan Pengabdian Masyarakat UNY: Yogyakarta.

Hanna, W. F., & Retnawati, H. (2022). Analisis Kualitas Butir Soal Matematika Menggunakan Model Rasch Dengan Bantuan Software Quest. AKSIOMA: Jurnal Program Studi Pendidikan Matematika, 11(4), 3695-3704.

Ghazali, N. H. M. (2016). A Reliability and Validity of an Instrument to Evaluate the School-Based Assessment System: A Pilot Study. International journal of evaluation and research in education, 5(2), 148-157.

Setyaningrum, P. M. P., Ramli, M., & Rinanto, Y. (2018). Analisis kualitas butir soal instrumen assessment diagnostic untuk mendeteksi miskonsepsi siswa SMA pada materi virus. Jurnal Bioedukatika, 6(2), 91-101.

Borualogo, I. S., Kusdiyati, S. K., Susandari, S. S., & Sirodj, D. A. N. (2017). Analisis item soal UTS pedologi semester ganjil 2015-2016. Schema: Journal of Psychological Research, 46-57.

Nofiana, M. (2017). Pengembangan instrumen evaluasi higher orderthinking skills pada materi kingdom plantae. Pedagogi Hayati, 1(1).

Arifin, Z. (2011). Evaluasi Pembelajaran. Remaja Rosdakarya.

Iskandar, A., & Rizal, M. (2018). Analisis kualitas soal di perguruan tinggi berbasis aplikasi TAP. Jurnal Penelitian Dan Evaluasi Pendidikan, 22(1), 12-23.

Khatmani, F. K., Liliawati, W., & Imansyah, H. Analisis pemahaman konsep fisika siswa smp pada materi suhu dan kalor menggunakan tes isomorfik: Rasch model. WaPFi (Wahana Pendidikan Fisika), 9(1), 11-18.

Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 020111.

Ardiyanti, D. (2016). Aplikasi model rasch pada pengembangan skala efikasi diri dalam pengambilan keputusan karir siswa. Jurnal Psikologi, 43(3), 248-263.

Pratama, D. (2020). Analisis kualitas tes buatan guru melalui pendekatan item response theory (IRT) model rasch. Tarbawy: Jurnal Pendidikan Islam, 7(1), 61-70.

Purba, S. E. D. (2018). Analisis model Rasch instrumen tes prestasi pada mata pelajaran dasar dan pengukuran listrik. Wiyata Dharma: Jurnal Penelitian Dan Evaluasi Pendidikan, 6(2), 142-147.

Hakiki, A. W., Fitri, A. R., & Agung, I. M. (2018). Analisis properti psikometri subtes merkaufgaben (ME) dengan rasch model. Jurnal Psikologi, 14(1), 40-49.

Istiyono, E., Mardapi, D., & Suparno, S. (2014). Pengembangan tes kemampuan berpikir tingkat tinggi fisika (pysthots) peserta didik SMA. Jurnal penelitian dan evaluasi pendidikan, 18(1), 1-12.

DOI: http://dx.doi.org/10.26737/jipf.v9i3.5692