Developing an Analytic Scale for Scoring EFL Descriptive Writing

Khatib, Mohammad; Mirzaii, Mostafa

Document Type : Research Paper

Authors

Mohammad Khatib ¹
Mostafa Mirzaii ²

¹ Associate Professor of TEFL, Allameh Tabataba'i University

² PhD Candidate of TEFL, Allameh Tabataba'i University

Abstract

English language practitioners have long relied on intuition-based scales for rating EFL/ESL writing. As these scales lack an empirical basis, the scores they generate tend to be unreliable, which results in invalid interpretations. Given the significance of the genre of description and the fact that the relevant literature does not introduce any data-based analytic scales for rating EFL descriptive writing, the researcher conducted a three-strand mixed study aimed at the empirical development of an analytic rating scale for scoring descriptions written by EFL learners. Composed of one quantitative and two qualitative strands, this mixed study factor-analyzed 172 ELT experts' analyses of the genre of description, and it content-analyzed 20 authentic and 30 inauthentic descriptive texts. Resulting from two meta-inferences made in the course of this study, the Analytic Rating Scale for EFL Descriptive Writing was constructed. Hopefully, employing this scale will lead to more reliable scores and more valid interpretations and decisions.

Keywords

References

Alexander, L. G. (1965). Essay and letter writing. London: Longman Publishing Group.

Bacha, N. (2001). Writing evaluation: What can analytic versus holistic scoring tell us? System, 29, 371-383.

Becker, A. (2011). Examining rubrics used to measure writing performance in U.S. intensive English programs. The CATESOL Journal, 22(1), 113-130.

Carr, N. (2000). A comparison of the effects of analytic and holistic rating scale types in the context of composition texts. Issues in Applied Linguistics, 11, 207-241.

Clare, A., & Wilson, J. J. (2005). Total English: Intermediate students’ book. London: Pearson Education Inc.

Cohen, A. (1994). Assessing language ability in the classroom. Boston, MA: Heinle & Heinle. The descriptive essay rubric. Retrieved June 2, 2012, from www.gnaedu.org. The descriptive writing rubric. Retrieved May 29, 2012, from msjohnsononline.weebly.com/uploads/.../1c_rubric_decriptivewritin. Pdf

East, M., & Young, D. (2007). Scoring L2 writing samples: Exploring the relative effectiveness of two different diagnostic methods. New Zealand Studies in Applied Linguistics, 13, 1-21.

Freedman, S. W. (1991). Evaluating writing: Linking large-scale testing and classroom assessment. Retrieved April 21, 2012, from www.nwp.org/cs/public/download/nwp_file/50/OP27.pdf?xr=pcfile

Fulcher, G., & Davidson, F. (2007). Language testing and assessment: An advanced resource book. New York: Taylor & Francis Group.

Hamp-Lyons, L. (2003). Writing teachers as assessors of writing. In B. Kroll (Ed.), Exploring the dynamics of second language writing (pp. 162-189). Cambridge: Cambridge University Press.

Hogue, A. (2008). First steps in academic writing (2nd ed.). New York: Pearson Education Inc.

Huot, B. (1990). Reliability, validity, and holistic scoring: What we know and what we need to know. College Composition and Communication, 41, 201-213.

Iwashita, N., & Grove, E. (2003). A comparison of analytic and holistic scales in the context of a specific purpose speaking test. Prospect, 18(3), 25-35.

Jacobs, H. L., Zinkgraf, D. R., Wormuth, D. R., Hartfiel, V. F., and Hughey, J. B. (1981). Testing ESL composition: A practical approach. Rowley, Massachusetts: Newbury House.

Knoch, U. (2007). Do empirically developed rating scales function differently to conventional rating scales for academic writing? Spaan Fellow Working Papers in Second or Foreign Language Assessment, 5, 1-36.

Knoch, U. (2009). Diagnostic assessment of writing: A comparison of two rating scales. Language Testing, 26, 275-304.

Meyers, A. (2006). Writing with confidence: Writing effective sentences and paragraphs (8th ed.). London: Pearson Education Inc.

Myford, C. M., & Wolfe, E. W. (2004). Detecting and measuring rater effects using many-faceted rasch measurement: Part 1. Journal of Applied Measurement, 4, 386-422.

Nakamura, Y. (2004). A comparison of holistic and analytic scoring methods in the assessment of writing. Retrieved March 10, 2012, from jalt.org/pansig/2004/HTML/Nakamura.html

Nelson, N. W., & Van Meter, A. M. (2007). Measuring written language ability in narrative samples. Reading & Writing Quarterly, 23, 287-309.

Nunnally, J. C. (1978). Psychometric theory. (2nd ed.). New York: McGraw-Hill.

Oshima, A., & Hogue, A. (2007). Introduction to academic writing. New York: Pearson Education Inc.

Pallant, J. (2013). A Step by step guide to data analysis using IBM SPSS: Survival manual. New York: The McGraw Hill Companies. The rubric for descriptive writing. Retrieved May 27, 2012, from www.pinterest.com/pin/285697170081236048/

Shaw, S. D., & Weir, C. J. (2007). Examining writing: Research and practice in second language writing. Cambridge: Cambridge University Press.

Soars, L., & Soars, J. (2009). New headway: Intermediate student's book (4th ed.). Oxford: Oxford University Press.

Song, B., & Caruso, I. (1996). Do English and ESL faculty differ in evaluating the essays of native English-speaking and ESL students? Journal of Second Language Writing, 5, 163-182.

Strauch, A. O. (1994). Writers at work: The short composition. Cambridge: Cambridge University Press.

Thurstone, L. L. (1947). Multiple factor analysis. Chicago: University of Chicago Press.

Weigle, S. C. (2002). Assessing writing. Cambridge: Cambridge University Press.

Weir, C. J. (1990). Communicative language testing. NJ: Prentice Hall Regents.

White, E. M. (1985). Teaching and assessing writing. San Francisco: Jossey-Bass.

Journal of English Language Teaching and Learning

Developing an Analytic Scale for Scoring EFL Descriptive Writing

References

References

East, M., & Young, D. (2007). Scoring L2 writing samples: Exploring the relative effectiveness of two different diagnostic methods. New Zealand Studies in Applied Linguistics, 13, 1-21.

Freedman, S. W. (1991). Evaluating writing: Linking large-scale testing and classroom assessment. Retrieved April 21, 2012, from www.nwp.org/cs/public/download/nwp_file/50/OP27.pdf?xr=pcfile

Fulcher, G., & Davidson, F. (2007). Language testing and assessment: An advanced resource book. New York: Taylor & Francis Group.

Hamp-Lyons, L. (2003). Writing teachers as assessors of writing. In B. Kroll (Ed.), Exploring the dynamics of second language writing (pp. 162-189). Cambridge: Cambridge University Press.

Hogue, A. (2008). First steps in academic writing (2nd ed.). New York: Pearson Education Inc.

Huot, B. (1990). Reliability, validity, and holistic scoring: What we know and what we need to know. College Composition and Communication, 41, 201-213.

Volume 8, Issue 17 - Serial Number 17
July 2016
Pages 49-73

Developing an Analytic Scale for Scoring EFL Descriptive Writing

References

References

East, M., & Young, D. (2007). Scoring L2 writing samples: Exploring the relative effectiveness of two different diagnostic methods. New Zealand Studies in Applied Linguistics, 13, 1-21.

Freedman, S. W. (1991). Evaluating writing: Linking large-scale testing and classroom assessment. Retrieved April 21, 2012, from www.nwp.org/cs/public/download/nwp_file/50/OP27.pdf?xr=pcfile

Fulcher, G., & Davidson, F. (2007). Language testing and assessment: An advanced resource book. New York: Taylor & Francis Group.

Hamp-Lyons, L. (2003). Writing teachers as assessors of writing. In B. Kroll (Ed.), Exploring the dynamics of second language writing (pp. 162-189). Cambridge: Cambridge University Press.

Hogue, A. (2008). First steps in academic writing (2nd ed.). New York: Pearson Education Inc.

Huot, B. (1990). Reliability, validity, and holistic scoring: What we know and what we need to know. College Composition and Communication, 41, 201-213.

Volume 8, Issue 17 - Serial Number 17July 2016Pages 49-73

Volume 8, Issue 17 - Serial Number 17
July 2016
Pages 49-73