Development of Physics HOTS Assessment Instruments in High School: A Comprehensive Literature Review

Yoga Budi Bhakti; Riyan Arthur; Yetti Supriyati

doi:10.26737/jipf.v10i2.5962

Development of Physics HOTS Assessment Instruments in High School: A Comprehensive Literature Review

Yoga Budi Bhakti, Riyan Arthur, Yetti Supriyati

Abstract

This study aims to identify strategies for developing HOTS assessment instruments in high school physics subjects. The research employs bibliometric methods and qualitative approach to analyze various journals published between 2019 and 2023. The analysis results indicate that the development of HOTS instruments is driven by the need for 21st-century skills and is supported by findings from international surveys such as PISA and TIMSS. The most commonly developed assessment forms include test questions in the form of multiple choice, multiple choice with reasoning, and essays. These questions are designed with HOTS characteristics in mind, including HOTS indicators, operational verbs, physics problem contexts, stimuli, and Bloom's taxonomy. Physics topics that are frequently developed include subjects such as temperature and heat, harmonic vibrations, sound and light waves, thermodynamics, as well as static and dynamic fluids. Commonly used research methods for this development include Borg and Gall's R&D method, the 4D method, qualitative descriptive methods, and the ADDIE method. HOTS instrument undergoes a series of feasibility tests and analyses, including validity testing by validators, statistical validity tests, reliability tests, difficulty level tests, discrimination index tests, Rasch model analysis, analysis using the Alpha Cronbach formula, and distractor analysis. The findings of this study provide valuable insights for developing valid, reliable, and relevant HOTS assessment instruments that meet the needs of high school physics education.

Keywords

Higher Order Thinking Skills; Physics Instrument; Assessment Instrument; Literature Review

Full Text:

PDF (English)

References

Katoningsih, S., & Sunaryo, I. (2020). Programme for International Student Assesment (PISA) as reading literacy standard: Critical thinking skill is priority. Education, Sustainability And Society, 3(1), 08-10.

Ramdhani, S. S., & Susanti, R. (2024). Cognitive level of program for international student assessment (PISA) questions based on the revised Bloom’s taxonomy. European Journal of Education and Pedagogy, 5(2), 104-112.

Assem, H. D., Nartey, L., Appiah, E., & Aidoo, J. K. (2023). A review of students’ academic performance in physics: Attitude, instructional methods, misconceptions and teachers qualification. European Journal of Education and Pedagogy, 4(1), 84-92.

Cai, S., Liu, C., Wang, T., Liu, E., & Liang, J. C. (2021). Effects of learning physics using Augmented Reality on students’ self‐efficacy and conceptions of learning. British Journal of Educational Technology, 52(1), 235-251.

Bhakti, Y. B., Arthur, R., & Supriyati, Y. (2024). The Content Validity: Multiple Choices Complex Physics Instrument to Measure Critical Thinking Skills. JIPF (Jurnal Ilmu Pendidikan Fisika), 9(1), 56-64.

Hyytinen, H., Holma, K., Toom, A., Shavelson, R. J., & Lindblom-Ylänne, S. (2014). The Complex Relationship between Students' Critical Thinking and Epistemological Beliefs in the Context of Problem Solving. Frontline Learning Research, 2(5), 1-25.

Eichenlaub, M., & Redish, E. F. (2019). Blending physical knowledge with mathematical form in physics problem solving. Mathematics in physics education, 127-151.

Bhakti, Y. B., Astuti, I. A. D., & Prasetya, R. (2023, July). Four-Tier Optics Diagnostic Test (4T-ODT) to Identify Student Misconceptions. In 3rd International Conference on Education and Technology (ICETECH 2022) (pp. 308-314). Atlantis Press.

Astutik, S., Bektiarso, S., Lesmono, A. D., & Nuraini, L. (2019, March). What can students show about higher order thinking skills in physics learning?. In IOP Conference Series: Earth and Environmental Science (Vol. 243, No. 1, p. 012127). IOP Publishing.

Danial, M., Yunus, M., & Syamsir, M. (2021, May). A Development of IPA (Natural Sciences) Learning Tools Based on Investigative Approach in Empowering Students’ Higher-Order Thinking Skills and Concept Mastery in Junior High School. In Journal of Physics: Conference Series (Vol. 1899, No. 1, p. 012143). IOP Publishing.

Jailani, J., Sugiman, S., & Apino, E. (2017). Implementing the problem-based learning in order to improve the students’ HOTS and characters. Jurnal Riset Pendidikan Matematika, 4(2), 247-259.

Anggraini, N. P., & Pratiwi, H. (2019, April). Analysis of higher order thinking skills students at junior high school in Surakarta. In Journal of Physics: Conference Series (Vol. 1211, No. 1, p. 012077). IOP Publishing.

Batlolona, J. R., & Dıantoro, M. (2019). Creative thinking skills students in physics on solid material elasticity. Journal of Turkish Science Education, 16(1), 48-61.

Kuo, E., Hull, M. M., Gupta, A., & Elby, A. (2013). How students blend conceptual and formal mathematical reasoning in solving physics problems. Science Education, 97(1), 32-57.

Affandy, H., Sunarno, W., & Suryana, R. (2024). Integrating creative pedagogy into problem-based learning: The effects on higher order thinking skills in science education. Thinking Skills and Creativity, 101575.

Hidayati, A., Munasir, M., & Sudibyo, E. (2024). Development of problem-based learning model physics learning tools to improve critical thinking skills of high school students. IJORER: International Journal of Recent Educational Research, 5(1), 77-89.

Zulyusri, Z., Elfira, I., Lufri, L., & Santosa, T. A. (2023). Literature study: Utilization of the PjBL model in science education to improve creativity and critical thinking skills. Jurnal Penelitian Pendidikan IPA, 9(1), 133-143.

Krathwohl, D. R. (2002). A Revision Bloom's Taxonomy: An Overview. Theory into Practice, 41(4), 212-218.

Jaenudin, R., Chotimah, U., Farida, F., & Syarifuddin, S. (2020). Student development zone: higher order thinking skills (hots) in critical thinking orientation. International Journal of Multicultural and Multireligious Understanding, 7(9), 11-19.

Jihannita, J., Prasetyo, Z. K., & Wilujeng, I. (2023). How to Prepare HOTS to Face the 21st Century?. Jurnal Penelitian Pendidikan IPA, 9(8), 486-492.

Ramadhan, S., Mardapi, D., Prasetyo, Z. K., & Utomo, H. B. (2019). The development of an instrument to measure the higher order thinking skill in physics. European Journal of Educational Research, 8(3), 743-751.

Yusuf, I., Wıdyanıngsıh, S. W., & Sebayang, S. R. B. (2018). Implementation of E-learning based-STEM on quantum physics subject to student HOTS ability. Journal of Turkish Science Education, 15(STEM Special Issue), 67-75.

Alanazi, A. A., Osman, K., & Halim, L. (2024). Effect of scaffolding strategies and guided discovery on higher-order thinking skills in physics education. EURASIA Journal of Mathematics, Science and Technology Education, 20(9), em2496.

Zulhelmi, Z., Fauza, N., Syaflita, D., Pratiwi, J., Wijaya, T. T., & Hermita, N. (2023). Development of Learning Media to Improve Students' Higher Order Thinking Skills in Circular Motion Material. Jurnal Penelitian Pendidikan IPA, 9(4), 1734-1740.

Arafah, K., Amin, B. D., Sari, S. S., & Hakim, A. (2021, May). The development of higher order-thinking skills (HOTS) instrument assessment in physics study. In Journal of Physics: Conference Series (Vol. 1899, No. 1, p. 012140). IOP Publishing.

Walid, A., Sajidan, S., Ramli, M., & Kusumah, R. G. T. (2019). Construction of the assessment concept to measure students' high order thinking skills. Journal for the Education of Gifted Young Scientists, 7(2), 237-251.

Ahmad, S., Prahmana, R. C. I., Kenedi, A. K., Helsa, Y., Arianil, Y., & Zainil, M. (2017, December). The instruments of higher order thinking skills. In Journal of Physics: Conference Series (Vol. 943, No. 1, p. 012053). IOP Publishing.

Merta Dhewa, K., Rosidin, U., Abdurrahman, A., & Suyatna, A. (2017). The development of Higher Order Thinking Skill (Hots) instrument assessment in physics study. IOSR Journal of Research & Method in Education (IOSR-JRME), 7(1), 26-32.

Istiyono, E. (2018, September). IT-based HOTS assessment on physics learning as the 21st century demand at senior high schools: Expectation and reality. In AIP Conference Proceedings (Vol. 2014, No. 1). AIP Publishing.

Jauhariyah, M. N. R., Sunarti, T., Setyarsih, W., & Zainuddin, A. (2021, March). Analysis of physics questions based on HOTS criteria: the result of physics teacher training. In Journal of Physics: Conference Series (Vol. 1805, No. 1, p. 012023). IOP Publishing.

Eka Mahendra, I. W. (2020). Teachers’ Formative Assessment: Accessing Students’ High Order Thinking Skills (HOTS)?. International Journal of Innovation, Creativity and Change, 12(12), 180-202.

Yusuf, I., Widyaningsih, S. W., Prasetyo, Z. K., & Istiyono, E. (2021, March). The evaluation on the use of e-learning media to improve HOTS through authentic and holistic assessments. In Journal of Physics: Conference Series (Vol. 1806, No. 1, p. 012014). IOP Publishing.

Sulisworo, D., & Basriyah, K. (2021, February). Problem based learning using open educational resources to enhance higher order thinking skills in physics learning. In Journal of Physics: Conference Series (Vol. 1783, No. 1, p. 012108). IOP Publishing.

Oyewola, D. O., & Dada, E. G. (2022). Exploring machine learning: a scientometrics approach using bibliometrix and VOSviewer. SN Applied Sciences, 4(5), 143.

Bukar, U. A., Sayeed, M. S., Razak, S. F. A., Yogarayan, S., Amodu, O. A., & Mahmood, R. A. R. (2023). A method for analyzing text using VOSviewer. MethodsX, 11, 102339.

Hartik, S., Utaminingsih, S., & Madjdi, A. H. (2021, March). A need assessment of integrated science teaching material based higher order thinking skills (hots). In Journal of Physics: Conference Series (Vol. 1823, No. 1, p. 012078). IOP Publishing.

Petrovska, S., & Veselinovska, S. S. (2013). Contemporary pedagogical approaches for developing higher level thinking on science classes. Procedia-Social and Behavioral Sciences, 92, 702-710.

Alexander, P. A., & Kulikowich, J. M. (1994). Learning from physics text: A synthesis of recent research. Journal of research in science teaching, 31(9), 895-911.

Zohar, A. (2013). Challenges in wide scale implementation efforts to foster higher order thinking (HOT) in science education across a whole school system. Thinking Skills and Creativity, 10, 233-249.

Zohar, A., & Schwartzer, N. (2005). Assessing teachers’ pedagogical knowledge in the context of teaching higher‐order thinking. International Journal of Science Education, 27(13), 1595-1620.

Illene, S., Feranie, S., & Siahaan, P. (2023). Create multiple-choice tests based on experimental activities to assess students' 21st century skills in the heat and heat transfer topic. Journal of Education and Learning (EduLearn), 17(1), 44-57.

Widyaningsih, S. W., Yusuf, I., Prasetyo, Z. K., & Istiyono, E. (2021). The Development of the HOTS Test of Physics Based on Modern Test Theory: Question Modeling through E-Learning of Moodle LMS. International Journal of Instruction, 14(4), 51-68.

Nurulsari, N., Rosidin, U., & Dwi Indriyani, E. V. (2021). Development of HOTS Assessment Instruments on Newton's Law of Motion by Using Multiple Types of Problems in High School Physics Lessons. Jurnal Pembelajaran Fisika, 9(2), 129-141.

Suprapto, E., Sumiharsono, R., & Ramadhan, S. (2020). The Analysis of Instrument Quality to Measure the Students' Higher Order Thinking Skill in Physics Learning. Journal of Turkish Science Education, 17(4), 520-527.

Maryani, I., Prasetyo, Z. K., Wilujeng, I., Purwanti, S., & Fitrianawati, M. (2021). HOTs Multiple Choice and Essay Questions: A Validated Instrument to Measure Higher-Order Thinking Skills of Prospective Teachers. Journal of Turkish Science Education, 18(4), 674-690.

Darmana, A., & Sutiani, A. (2020, February). Development of The Thermochemistry-Hots-Tawheed Multiple Choice Instrument. In Journal of Physics: Conference Series (Vol. 1462, No. 1, p. 012057). IOP Publishing.

Megawan, M., & Istiyono, E. (2019, June). Physics creative thinking measurement using two-tier multiple choice to support science, technology, engineering, and mathematics. In Journal of Physics: Conference Series (Vol. 1233, No. 1, p. 012068). IOP Publishing.

Istiyono, E. (2019, June). Constructing Reasoning Multiple Choice Test to Measure Bloomian Higher Order Thinking Skills in Physics of XI Grade Students. In Journal of Physics: Conference Series (Vol. 1233, No. 1, p. 012037). IOP Publishing.

Istiyono, E., Dwandaru, W. S. B., Fenditasari, K., Ayub, M. R. S. S. N., & Saepuzaman, D. (2023). The Development of a Four-Tier Diagnostic Test Based on Modern Test Theory in Physics Education. European Journal of Educational Research, 12(1).

Erfianti, L., Istiyono, E., & Kuswanto, H. (2019). Developing lup instrument test to measure higher order thinking skills (HOTS) Bloomian for senior high school students. International Journal of Educational Research Review, 4(3), 320-329.

Rintayati, P., Lukitasari, H., & Syawaludin, A. (2021). Development of Two-Tier Multiple Choice Test to Assess Indonesian Elementary Students' Higher-Order Thinking Skills. International Journal of Instruction, 14(1), 555-566.

Sumarni, S. (2020). Development and Validation of Open Ended Based on Worksheet for Growing Higher Level Thinking Skills of Students. European Journal of Educational Research, 9(2,445-455).

Romli, S., Distrik, I. W., Herlina, K., Umam, R., Ramadhani, R., & Sumarni, S. (2020). Development and Validation of Open Ended Based on Worksheet for Growing Higher Level Thinking Skills of Students. European Journal of Educational Research, 9(2), 445-455.

Sunarti, T., & Jauhariyah, M. N. R. (2021, March). Train the skills of making HOTS-based physics questions to physics teachers in Mojokerto. In Journal of Physics: Conference Series (Vol. 1805, No. 1, p. 012027). IOP Publishing.

Zaki, M., Amalia, R., & Sofyan, S. (2020, April). Development of high order thinking skills (HOTS) test instrument on exponent for junior high school students. In Journal of Physics: Conference Series (Vol. 1521, No. 3, p. 032096). IOP Publishing.

Fanani, R. D., Supardi, Z. A. I., & Suprapto, N. (2023). The Development of HOTS-based Question Instrument in Temperature and Heat Material. Jurnal Penelitian Pendidikan IPA, 9(9), 6890-6895.

Retnawati, H., Djidu, H., Apino, E., & Anazifa, R. D. (2018). Teachers' Knowledge about Higher-Order Thinking Skills and Its Learning Strategy. Problems of Education in the 21st Century, 76(2), 215-230.

Takko, M., Jamaluddin, R., Kadir, S. A., Ismail, N., Abdullah, A., & Khamis, A. (2020). Enhancing higher-order thinking skills among home science students: The effect of cooperative learning student teams-achievement divisions (STAD) module. International Journal of Learning, Teaching and Educational Research, 19(7), 204-224.

Rifâ, A., Serevina, V., & Delina, M. (2018). The development of High Order Thinking Skills (HOTS) assessment instrument for temperature and heat learning. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 4(1), 19-26.

Istikomah, I., Suyatna, A., Viyanti, V., & Andra, D. (2021). The Development of Computer Based Test Instruments using Wondershare Quiz Creator to Measure HOTS in Temperature and Heat. Jurnal Pendidikan MIPA, 22(1), 45-51.

Akhsan, H., Wiyono, K., Ariska, M., & Melvany, N. E. (2020, March). Development of HOTS (higher order thinking skills) test instruments for the concept of fluid and harmonic vibrations for high schools. In Journal of Physics: Conference Series (Vol. 1480, No. 1, p. 012071). IOP Publishing.

Astra, I. M., Susila, A. B., & Sofiana, M. (2019). Development of an electronic module with a discovery learning model to improve Higher-Order Thinking Skills (HOTS) learning outcomes in a senior high school study of harmonic oscillation. In Empowering Science and Mathematics for Global Competitiveness (pp. 308-313). CRC Press.

Zahra, S., & Suwarna, I. P. (2023). Exploration of Increasing Students’ Critical Thinking in the Guided Discovery Learning Model on Light Waves. Jurnal Ilmiah Pendidikan Fisika, 7(2), 354-368.

Agustihana, S., & Suparno, S. (2019). Development of HOTS Oriented Cognitive Problems in Thermodynamics for Senior High Schools. Jurnal Penelitian Fisika Dan Aplikasinya (JPFA), 9(1), 44-54.

Saefullah, A., Fitriyani, A., Ruhiyat, Y., & Rostikawati, D. A. (2020). Blended Learning: The Effect on Higher Order Thinking Skills (HOTS) in Thermodynamics. Indonesian Journal of Science and Mathematics Education, 3(3), 262-271.

Linuwih, S., & Safutra, Y. (2023, June). Development of HOTS questions on first law of thermodynamics materials to measure higher order thinking skills for SMAN 1 Jakenan students. In AIP Conference Proceedings (Vol. 2614, No. 1). AIP Publishing.

Serevina, V., Sari, Y. P., & Maynastiti, D. (2019, April). Developing high order thinking skills (HOTS) assessment instrument for fluid static at senior high school. In Journal of Physics: Conference Series (Vol. 1185, No. 1, p. 012034). IOP Publishing.

Pratiwi, D. E., Suyatna, A., & Viyanti, V. (2020, February). Design of Computer-Based Testing for Higher-order Thinking Skills on Static Fluid Material. In Journal of Physics: Conference Series (Vol. 1467, No. 1, p. 012059). IOP Publishing.

Sari, S., & Suyatna, A. (2021, February). Need assesment and design of e-modules to stimulate HOTS on dynamic fluid materials with the STEM approach. In Journal of Physics: Conference Series (Vol. 1796, No. 1, p. 012003). IOP Publishing.

Qotrunnada, N. A., & Prahani, B. K. (2022). Profile of PBL model assisted by digital books to improve problem solving ability of high school students on dynamic fluids. Jurnal Penelitian Pendidikan IPA, 8(3), 1175-1183.

Al Fath, R., & Dewi, W. S. (2022). The Effect of Student Worksheets Based on Problem Solving on Students' Critical Thinking Ability in Class XI SMA on Dynamic Fluid Materials. Pillar of Physics Education, 15(2), 138-147.

Yusuf, I., Widyaningsih, S. W., Prasetyo, Z. K., & Istiyono, E. (2020, April). Higher order thinking skills (HOTS)-oriented e-module in electric circuit. In Journal of Physics: Conference Series (Vol. 1521, No. 2, p. 022027). IOP Publishing.

Kristiani, K., & Sulistyaningrum, C. D. (2024). Development of hots-based cognitive assessment instruments: ADDIE model. Journal of Education and Learning (EduLearn), 18(2), 489-498.

Milia, A. H., Trisna, S., & Yanti, I. R. (2022). Development of HOTS assessment instruments on static electricity materials for high school level. Berkala Ilmiah Pendidikan Fisika, 10(1), 73-83.

Dasilva, B. E. (2019, December). Development of the Android-Based Interactive Physics Mobile Learning Media (IPMLM) to Improve Higher Order Thinking Skills (HOTS) of Senior High School Students. In Journal of Physics: Conference Series (Vol. 1397, No. 1, p. 012010). IOP Publishing.

Istiyono, E., Dwandaru, W. S. B., Setiawan, R., & Megawati, I. (2020). Developing of Computerized Adaptive Testing to Measure Physics Higher Order Thinking Skills of Senior High School Students and Its Feasibility of Use. European Journal of Educational Research, 9(1), 91-101.

Reeves, T. D., & Marbach-Ad, G. (2016). Contemporary test validity in theory and practice: A primer for discipline-based education researchers. CBE—Life Sciences Education, 15(1), rm1.

Hajjar, S. T. (2018). Statistical analysis: Internal-consistency reliability and construct validity. International Journal of Quantitative and Qualitative Research Methods, 6(1), 27-38.

Burud, I., Nagandla, K., & Agarwal, P. (2019). Impact of distractors in item analysis of multiple choice questions. International Journal of Research in Medical Sciences, 7(4), 1136-1139.

D'Sa, J. L., & Visbal-Dionaldo, M. L. (2017). Analysis of multiple choice questions: item difficulty, discrimination index and distractor efficiency. International Journal of Nursing Education, 9(3).

Zamanian, M., & Heydari, P. (2012). Readability of Texts: State of the Art. Theory & Practice in Language Studies (TPLS), 2(1).

Azevedo, J. M., Oliveira, E. P., & Beites, P. D. (2019). Using learning analytics to evaluate the quality of multiple-choice questions: A perspective with classical test theory and item response theory. The International Journal of Information and Learning Technology, 36(4), 322-341.

Bichi, A. A. (2016). Classical Test Theory: an introduction to linear modeling approach to test and item analysis. International Journal for Social Studies, 2(9), 27-33.

DOI: http://dx.doi.org/10.26737/jipf.v10i2.5962