The Design and Validation of the Four Tier Test Instrument for Energy Literacy Using the Rasch Model Analysis

Tri Hastiti Fiskawarni; Rahmawati Rahmawati; Widiasih Widiasih; Rezkawati Saad

doi:10.26737/jipf.v9i1.4717

The Design and Validation of the Four Tier Test Instrument for Energy Literacy Using the Rasch Model Analysis

Tri Hastiti Fiskawarni, Rahmawati Rahmawati, Widiasih Widiasih, Rezkawati Saad

Abstract

The science education curriculum needs to contain content on environmental issues including energy and its use as an effort to equip prospective physics teacher students with knowledge about the importance of reducing the impact of energy use. For this reason, it is important to measure energy literacy knowledge in prospective physics teacher students. This study aims to design and validate a four-tier test instrument to measure the energy literacy knowledge of prospective physics teacher students. The test instrument format used is a four-tier test. This test model has the advantage of being able to capture more accurate information with various answer patterns. The stages of developing this test used the design-based research model which consisted of five stages, namely developing an assessment framework, designing items, developing rubrics, conducting tests, and applying the Racsh Model analysis. The application of the Racsh Model analysis aims to obtain a valid and reliable test instrument with the Item Response Theory (IRT) approach assisted by the Winsteps program. The research method used is a descriptive-exploratory method to describe the results of the development and validation of the Four Tier test to measure Energy Literacy for prospective physics teacher student. The validation of the test was carried out through an assessment by five experts to assess the construct and content of the test instrument. The results of the item validation showed that the questions were acceptable in all aspects. The conclusion is that the test with four-tier format is suitable for identifying the knowledge of prospective physics teacher students about Energy Literacy. The four tier test model in exploring the energy literacy abilities of prospective teacher students can basically also be applied to students at the elementary school, middle school and high school levels. However, the complexity of the content tested needs to be adjusted to the existing curriculum at each level.

Keywords

Energy literacy; Item response theory, Rasch model; Energy topic; Winstep version 3. 68. 2.

Full Text:

PDF (English)

References

Duit, R., Schecker, H., Höttecke, D., & Niedderer, H. (2014). Teaching physics. In Handbook of Research on Science Education, Volume II (pp. 448-470). Routledge.

Tajudin, N. A. M., & Chinnappan, M. (2015). Exploring Relationship between Scientific Reasoning Skills and Mathematics Problem Solving. Mathematics Education Research Group of Australasia.

Wellington, J. (2003). Science education for citizenship and a sustainable future. Pastoral Care in Education, 21(3), 13-18.

Hoque, F., Yasin, R. M., & Sopian, K. (2022). Revisiting education for sustainable development: Methods to inspire secondary school students toward renewable energy. Sustainability, 14(14), 8296.

Khushik, F., & Diemer, A. (2018). Critical analysis of education policies in Pakistan: A sustainable development perspective. Social Science Learning Education Journal, 3(09), 01-16.

Eilks, I. (2015). Science education and education for sustainable development–justifications, models, practices and perspectives. Eurasia Journal of Mathematics, Science and Technology Education, 11(1), 149-158.

Glavič, P. (2020). Identifying key issues of education for sustainable development. Sustainability, 12(16), 6500.

Zografakis, N., Menegaki, A. N., & Tsagarakis, K. P. (2008). Effective education for energy efficiency. Energy Policy, 36(8), 3226-3232.

Martín-Gámez, C., & Erduran, S. (2018). Understanding argumentation about socio-scientific issues on energy: a quantitative study with primary pre-service teachers in Spain. Research in Science & Technological Education, 36(4), 463-483.

Aguirre-Bielschowsky, I., Lawson, R., Stephenson, J., & Todd, S. (2017). Energy literacy and agency of New Zealand children. Environmental Education Research, 23(6), 832-854.

Chen, K. L., Liu, S. Y., & Chen, P. H. (2015). Assessing multidimensional energy literacy of secondary students using contextualized assessment. International Journal of Environmental and Science Education, 10(2), 201-218.

Kaltakci-Gurel, D., Eryilmaz, A., & McDermott, L. C. (2017). Development and application of a four-tier test to assess pre-service physics teachers’ misconceptions about geometrical optics. ReseaRch in science & Technological educaTion, 35(2), 238-260.

Fraenkel, J., Wallen, N., & Hyun, H. (2018). How to design and evaluate research in education (10th) ed.). McGraw-Hill.

Khaeruddin, K., Rahmawati, R., Nurfazlina, N., Salwa, R., & Nurhayati, N. (2022). The Development of Students’ Worksheets Face to Face Online Based on Hypercontent on Temperature and Heat Topic. Jurnal Penelitian Pendidikan IPA (JPPIPA), 8(6), 3011-3019.

Stephens, J. C., Hernandez, M. E., Román, M., Graham, A. C., & Scholz, R. W. (2008). Higher education as a change agent for sustainability in different cultures and contexts. International journal of sustainability in higher education, 9(3), 317-338.

Gänswein, W. (2011). Effectiveness of information use for strategic decision making. Wiesbaden: Gabler.

Yusup, M., Setiawan, A., Rustaman, N. Y., & Kaniawati, I. (2017, July). Developing a framework for the assessment of pre-service physics teachers’ energy literacy. In Journal of Physics: Conference Series (Vol. 877, No. 1, p. 012014). IOP Publishing.

Chen, S. J., Chou, Y. C., Yen, H. Y., & Chao, Y. L. (2015). Investigating and structural modeling energy literacy of high school students in Taiwan. Energy Efficiency, 8, 791-808.

Fell, M. J., & Chiu, L. F. (2014). Children, parents and home energy use: Exploring motivations and limits to energy demand reduction. Energy Policy, 65, 351-358.

Davis, P. (1985). The attitude and knowledge of tasmanian secondary students towards energy conservation and the environment. Research in Science Education, 15(1), 68-75.

DeWaters, J., & Powers, S. (2013). Establishing measurement criteria for an energy literacy questionnaire. The Journal of Environmental Education, 44(1), 38-55.

Halder, P., Pietarinen, J., Havu-Nuutinen, S., Pöllänen, S., & Pelkonen, P. (2016). The Theory of Planned Behavior model and students' intentions to use bioenergy: A cross-cultural perspective. Renewable Energy, 89, 627-635.

E Gronlunds, N. (2021). Measurement and assessment in teaching. Pakistan Journal of Educational Research and Evaluation (PJERE), 5(2).

Doran, R. L. (1980). Basic Measurement and Evaluation of Science Instruction. National Science Teachers Association, 1742 Connecticut Ave., NW, Washington, DC 20009 (Stock No. 471-14764; no price quoted)..

Engeström, Y., Virkkunen, J., Helle, M., Pihlaja, J., & Poikela, R. (1996). The change laboratory as a tool for transforming work. Lifelong learning in Europe, 1(2), 10-17.

Gronlund, N. E. (1998). Assessment of student achievement. Allyn & Bacon Publishing, Longwood Division, 160 Gould Street, Needham Heights, MA 02194-2310; tele.

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory (Vol. 2). Sage.

Retnawati, H. (2016). Validitas Reliabilitas & Karakteristik Butir (Panduan untuk Peneliti, Mahasiswa, dan Psikometrian) berbasis software. Nuha Medika.

DeMars, C. (2010). Item response theory. Oxford University Press.

Hambleton, R. K., & Swaminathan, H. (2013). Item response theory: Principles and applications. Springer Science & Business Media.

Kuo, C. Y., Wu, H. K., Jen, T. H., & Hsu, Y. S. (2015). Development and validation of a multimedia-based assessment of scientific inquiry abilities. International Journal of Science Education, 37(14), 2326-2357.

Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel psychology, 28(4), 563-575.

Wilson, F. R., Pan, W., & Schumsky, D. A. (2012). Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and evaluation in counseling and development, 45(3), 197-210.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan rasch pada assessment pendidikan. Trim komunikata.

Boone, W. J., Staver, J. R., & Yale, M. S. (2013). Rasch analysis in the human sciences. Springer Science & Business Media.

Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multiple‐choice tests. Science Education, 90(2), 253-269.

Davidowitz, B., & Potgieter, M. (2016). Use of the Rasch measurement model to explore the relationship between content knowledge and topic-specific pedagogical content knowledge for organic chemistry. International Journal of Science Education, 38(9), 1483-1503.

Bloom, B. S. (2010). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Longman.

Marzano, R. J. (2006). Classroom assessment & grading that work. ASCD.

Wilson, M. (2023). Constructing measures: An item response modeling approach. Taylor & Francis.

Gurcay, D., & Gulbas, E. (2015). Development of three-tier heat, temperature and internal energy diagnostic test. Research in Science & Technological Education, 33(2), 197-217.

Bansilal, S. (2015). A Rasch analysis of a Grade 12 test written by mathematics teachers. South African Journal of Science, 111(5-6), 1-9.

E Gronlunds, N. (2021). Measurement and assessment in teaching. Pakistan Journal of Educational Research and Evaluation (PJERE), 5(2).

Thorndike, R. L. (1982). Educational measurement: Theory and practice. The improvement of measurement in education and psychology, 3-13.

Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of applied measurement, 3(1), 85-106.

Linacre, J. M. (2010). Predicting responses from Rasch measures. Journal of Applied Measurement, 11(1), 1.

Bond, T. G., & Fox, C. M. (2013). Applying the Rasch model: Fundamental measurement in the human sciences. Psychology Press.

Krishnan, S., & Idris, N. (2014). Investigating reliability and validity for the construct of inferential statistics. International Journal of Learning, Teaching and Educational Research, 4(1), 51-60.

Sumintono, B., & Widhiarso, W. (2014). Aplikasi model Rasch untuk penelitian ilmu-ilmu sosial (edisi revisi). Trim Komunikata Publishing House.

Sumintono, B. (2017). Rasch Model Measurement as Tools in Assessment for Learning.

DOI: http://dx.doi.org/10.26737/jipf.v9i1.4717