Document Type : Research Paper
Author
Department of Language and Literature, Yazd University, Iran
Abstract
The field of language assessment, commemorating its 40th anniversary since the launch of language testing, has evolved significantly over the years. This study aimed to investigate the key findings and insights from exploring the role of construct validity in shaping the design of English Language Assessment (ELA) tasks. Additionally, it delved into the challenges encountered in construct validity research studies and the strategies suggested by experts to enhance it. The research team utilized a mixed-method research design for the current study. A total sample size of 37 participants was deployed. Descriptive statistics was used to summarize survey responses using quantitative analysis software (e.g., SPSS). Qualitative data was coded and organized using qualitative analysis software (e.g., NVIVO). Based on the research findings, experts in the current study have proposed strategies, and recommendations for enhancing construct validity. These strategies encompassed the incorporation of contextual factors into assessment design, the promotion of continuous validation research, the diversification of task types, and the active involvement of test-takers in the assessment development process. The findings of this study may render implications for EFL teachers, teacher trainers, and assessment administrators.
Keywords
- assessment design
- construct validity
- English language assessment (ELA)
- language assessment experts
- validity in assessment
Main Subjects
Ahmad, S., Sultana, N., & Jamil, S. (2020). Behaviorism vs constructivism: A paradigm shifts from traditional to alternative assessment techniques. Journal of Applied Linguistics and Language Research, 7(2), 19-33.
Ary, D., Jacobs, L. C., Irvine, C. K. S., & Walker, D. (2018). Introduction to research in education. Cengage Learning.
Aryadoust, V., Zakaria, A., Lim, M. H., & Chen, C. (2020). An extensive knowledge mapping review of measurement and validity in language assessment and SLA research. Frontiers in Psychology, 11, 1941. https://doi.org/10.3389/fpsyg.2020.01941
Azizi, Z. (2022). Fairness in assessment practices in online education: Iranian University English teachers’ perceptions. Language Testing in Asia, 12(14), 1-17. https://doi.org/10.1186/s40468-022-00164-7
Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford University Press.
Baker, B. A., & Riches, C. (2018). The development of EFL examinations in Haiti: Collaboration and language assessment literacy development. Language Testing, 35(4), 557–581.
Berry, V., Sheehan, S., & Munro, S. (2019). What does language assessment literacy mean to teachers? ELT Journal, 73(2), 113–123.
Borsboom, D., & Wijsen, L. D. (2016). Frankenstein’s validity monster: The value of keeping politics and science separated. Assessment in Education: Principles, Policy & Practice, 23(2), 281-283. https://doi.org/10.1080/0969594X.2016.1141750
Camilli, G., & Shepard, L. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
Chalhoub-Deville, M., & O’Sullivan, B. (2020). Validity: Theoretical development and integrated arguments. British Council Monographs.
Chapelle, C., & Voss, E. (Eds.) (2021). Validity argument in language testing: Case studies of validation research. Cambridge University Press.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302. https://psycnet.apa.org/doi/10.1037/h0040957
Demir, M., Tananis, C. A., & Trahan, K. W. (2019). Evaluation of alternative assessment methods used in elementary schools. Egitim ve Bilim, 44(197)
Dhindsa, H. S., Omar, K., & Waldrip, B. (2007). Upper secondary Bruneian science students’ perceptions of assessment. International Journal of Science Education, 29(10), 1261-1280.
Dobakhti, L. (2020). The process of enhancing validity, reliability, and ethics in research. Iranian Journal of Applied Language Studies, 12(2), 59-88. 10.22111/IJALS.2020.5978
Elharrar, Y. (2006). Teacher assessment practices and perceptions: the use of alternative assessments within the Quebec educational reform. Unpublished thesis. Montreal: Université du Québec à Montréal.
Fan, X., Liu, X., & Johnson, R. (2020). A mixed method study of ethical issues in classroom assessment in Chinese higher education. Asia Pacific Education Review, 21, 183-195. https://doi.org/10.1007/s12564-019-09623-y
Fraenkel, R. J., Wallen, E. N., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill
Giraldo, F. (2020). A post-positivist and interpretive approach to researching teachers’ language assessment literacy. Profile: Issues in Teachers’ Professional Development, 22(1), 189–200.
Giraldo, F. (2021). A reflection on initiatives for teachers’ professional development through language assessment literacy. Profile: Issues in Teachers’ Professional Development, 23(1), 197–213.
Harding, L., & Brunfaut, T. (2020). Trajectories of language assessment literacy in a teacher-researcher partnership: Locating elements of praxis through narrative inquiry. In: Poehner, M. E. and Inbar-Lourie, O. (Eds.), Towards a re-conceptualization of second language classroom assessment (pp. 61–81). Springer International Publishing.
Hasrol, B. S., Zakaria, A., & Aryadoust, V. (2022). A systematic review of authenticity in language assessment. Research Methods in Applied Linguistics, 1(3), 100023. https://doi.org/10.1016/j.rmal.2022.100023
Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-333.
Herman, J., & Cook, L. (2019). Fairness in classroom assessment. In S. M. Brookhart & J. H. McMillan (Eds.), Classroom assessment and educational measurement (pp. 243-264). Routledge.
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLOS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0040168
Isbell, D. R., Kremmel, B., & Kim, J. (2023). Remote proctoring in language testing: Implications for validity, fairness, and justice. Language Assessment Quarterly, 20(5), 469–487.
Jiang, Y. C., Jong, S. Y., Lau, W. F., Chai, C. S., Liu, S. X., & Park, M. Y. (2022). A scoping review on flipped classroom approach in language education: Challenges, implications and an interaction model. Computer Assisted Language Learning, 35(6), 1218–1249.
Kunnan, A. J. (2018). Evaluating language assessments. Routledge.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-104). American Council on Education and Macmillan.
Messick, S. (1975). The standard problem: Meaning and values in measurement and evaluation. American Psychologist, 30(10), 955–966. https://doi.org/10.1037/0003-066X.30.10.955
Nematzadeh, A. (2018). Construct Irrelevant Factors and Test Validity: Investigating the Relationship among Gender, Age, Mother Tongue, Field of Study and TOEFL IBT ® Results. Journal of Foreign Language Research, 8(1), 139-166. doi: 10.22059/jflr.2018.242996.405
Ockey, G. J., Chukharev-Hudilainen, E., & Hirch, R. R. (2023). Spoken dialogue systems and their potential for aiding in the assessment of interactional competence. Language Assessment Quarterly, 20(5), 377–398.
Tierney, R. (2016). Fairness in educational assessment. In M. A. Peters (Ed.). Encyclopedia of educational philosophy and theory (pp. 1-6). Springer Singapore.
Xu, Y., & Brown, G. (2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58, 149-162. https://doi.org/10.1016/j.tate.2016.05.010
Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393
Zohrabi, M. & Nasirfam, F. (2024). The use of assessment for learning rather than assessment of learning in EFL context. Applied Research on English Language, 13(2), 1-30. https://doi.org/10.22108/are.2024.140274.2210
)