Document Type : Research Paper

Authors

1 Assistant Professor, University of Human Development, College of Languages, English Department, Kurdistan, Iraq

2 Assistant Professor, Shahid Chamran University of Ahvaz, Ahvaz, Iran

3 Assistant Professor, Faculty of Humanities, Islamic Azad University of Ramhormoz, Ramhormoz, Iran

Abstract

Most of the validation studies conducted across varying test application contexts are usually framed within the traditional conceptualization of validity and therefore lack a comprehensive framework to focus on test score interpretations and test score use. This study aimed at developing and validating a collocational behavior test (CBT), drawing on Kane's argument-based approach to validity. Four types of inferences including observation, generalization, extrapolation and explanation were articulated. To verify the validity assumptions, both theoretical and empirical evidence were presented within the formative and summative stages of test development and validation. Followed from Kane, theoretical evidence was sought through test specification, item construction, and test construction procedures. Empirical support, however, was sought through examining the collocational behavior test (CBT) with a sample of 60 university students majoring in TEFL. Ebel’s criteria, KR-21 reliability and a series of Pearson-Product correlation were applied to analyze the data for both theoretical and empirical phases. The findings refer to the support for the assumptions proposed for test validity, suggesting that the collocational behavior test(CBT)may provide an appropriate and accurate indicator of collocational language ability for EFL learners. The implications for language testing and assessment are discussed

Keywords

Abdul Kadir, K. (2008). Framing a validity argument for test use and impact: Malaysian     public service       experience. Unpublished Doctoral dissertation, University of Illinois at          Urbana-Champaign.
Almela, M. (2007). Words as “lexical units” in learning teaching vocabulary. International   Journal of English Studies, 7(2), 21-40.
Aryadoust, V. (2011).validity arguments of the speaking and listening modules of    international English       language testing system: A synthesis of existing research. Asian         ESP Journal.7(2), 28-54.
Bachman, L. F., (1990). Fundamental considerations in language testing. Oxford: Oxford   University Press.
Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment    Quarterly, 2(1), 1-34.
Barfield, A. (2003). Collocation recognition and production: Research insights.  Tokyo:      Chuo University.
Bonk, W.J. (2001). Testing ESL learners’ knowledge of collocations. In T. Hudson &J. D. Brown (Eds.), A focus on language test development: expanding the language proficiency          construct across a variety of    tests. (Technical  eport#21). Honolulu: University of      Hawai’i, Second Language Teaching and Curriculum Center, (pp. 113-142).
Chapelle, C. A. (1994). Are C-tests valid measures for L2 vocabulary research? Second       Language Research,       10(2),   157-187.
Chapelle, C. A., Enright, M. K., & Jamieson, J. (2004). Issues in developing a TOEFL          validity argument: Paper presented at the 26th Annual Language Testing       Research          Colloquium, Temecula, CA.
Chapelle, C. A., Enright, M. K., & Jamieson, J. (2007a). (Where) is the construct in an         Interpretive Argument? Paper presented at the 29th Annual Language Testing Research Colloquium, Barcelona, Spain.
Chapelle, C. A., Enright, M. K., & Jamieson, J. (2007b). From validation research to a        validity argument. Paper presented at the 4th European Association for Language    Testing and Assessment Conference,       Sitges, Spain. 
Chapelle, C. A., Enright, M. E., & Jamieson, J. (Eds.) (2008). Building a validity argument   for the Test of       English as a Foreign Language. London: Routledge.
Chapelle, C. A., Enright, M. E., & Jamieson, J. (2010). Does an argument-based approach to            validity make a difference? Educational Measurement: Issues and Practice, 29(1), 3-    13.
Chapelle, C. A., & Read, J. (2001). A framework for vocabulary assessment. Language       Testing, 18(1), 1-32.
Cronbach, L. J. (.1971). Validity. In R. L. Thorndike (Ed.), Educational measurement (2nd    ed., pp. 443-597). Washington, DC: American Council on Education.
Fulcher, G., & Davidson, F. (2007). Language testing and assessment: An advanced resource         book. Oxford,       UK: Routledge.
House, E. R. (1980). Evaluating with validity. Beverly Hills, CA: SAGE.
Jaen, M.M. (2007). A corpus-driven design of a test for assessing the ESL collocational        competence of       university students. International journal of English Studies, 7(2), 127-         147.
 Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112,  527-535.
Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational       Measurement, 38, 319-342.
Kane, M. (2002). Validating high-stakes testing programs. Educational Measurement:  Issues           and Practice, 21(1), 31-41.
Kane, M. (2006). Validation. In R. Brennen (Ed.), Educational measurement (4th ed.), (pp.  17-64). Westport, CT: Greenwood.
Kane, M. T. (2006a). Content-related validity evidence. In S. M. Downing & T. M. Haladyna           (Eds.), Handbook of test development (pp. 131–154). Mahwah, NJ: Lawrence     Erlbaum.
Kane, M. T. (2006b). Validation. In R. L. Brennan (Ed.), Educational measurement (pp. 17- 64). American       Council on Education/Praeger.
Kane, M.T.(2011).Validating score interpretations and uses. Language Testing, 29(1) 3– 17
Keshavarz, M. H. & Salimi, H. (2007). Collocational competence and cloze test        performance: A study of Iranian EFL learners, 17(1), 81-92.
Le, H.T. (2011). Developing a validity argument for the English placement Fall 2010           Listening test at Iowa       State University, Unpublished Doctoral Dissertation, Iowa      State University.
McNamara, T. F., &Roever, C. (2006). Language testing: The social dimension. Maiden,     MA: Blackwell Publishing.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp.  13 103). New York: American Council on Education and Macmillan
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessment. Measurement: Interdisciplinary Research and Perspectives, 1, 3-62.
Mochizuki, M. (2002). Exploration of two aspects of vocabulary knowledge:  Paradigmatic  and collocational. Annual Review of English Language Education in Japan, 13, 121-            129.
Park, J. (2012). Developing and Validating an Instrument to Measure College Students’       Inferential Reasoning   in Statistics: An Argument-Based Approach to Validation.      Unpublished Doctoral dissertation, University of Minnesota.
Pawley, A., &Syder, F. H. (1983). Two puzzles for linguistic theory: Native like selection                 and native like  fluency. In J. C. Richards & R. W. Schmidt (Eds.), Language and     communication (pp. 191-226). New York, NY: Longman.
Qian, D. &Schedl, M. (2004). Evaluation of an in-depth vocabulary knowledge measure for assessing reading performance. Language Testing, 21(1), 28-52.
Johnson, R.C &Riazi, M.(2013). Assessing the assessments: Using an argument-based          validity framework to    assess the validity and use of an English placement system in a          foreign language context. Papers in Language Testing and Assessment, 2(1), 31-58.
Toulmin, S. E. (2003). The uses of argument (updated edition). Cambridge, UK:  Cambridge            University Press.
Voss, E. (2012). A validity argument for score meaning of a computer-based ESL academic collocational ability       test based on a corpus-driven approach to test design.           Unpublished doctoral dissertation, Iowa State University.
Widdowson, H. G., (2007). Discourse analysis. Oxford: Oxford University Press.