A Comparative Study of Writing Assessment Using Activity Theory-Based Assessment Model (ATBAM) and a Traditional Approach

Document Type: Research Paper

Authors

1 Associate Professor, University of Tehran

2 Assistant Professor, University of Tehran

3 PhD Candidate, University of Tehran, Alborz Campus

Abstract

Assessment of writing skill is generally believed to be judged by a rater subjectively and qualitatively or by using analytic scoring rubrics which can potentially result in somehow not very reliable assessment. It seems that an evaluation of writing based on a model can result in a valid and reliable writing assessment. To achieve such an objective, this study firstly aimed to develop an assessment model based on Activity Theory (AT), i. e., Activity Theory-Based Assessment Model (ATBAM), and then to employ it in the assessment of writing performances of Iranian language learners in a private language college. And finally, to achieve the concurrent validity of ATBAM, its results were compared with those of a traditional approach. Three groups of participants took part in this study: a group of upper intermediate English learners (N=29) who submitted one writing sample per week in four successive weeks, teachers (N=6) who provided learners with feedback and assigned holistic scores and course supervisors (N=2) who reassessed the writing samples on the basis of an analytic rubric of writing assessment.  The results showed that using ATBAM in writing assessment results in an exploration of not only learners’ but also teachers’ engagement in the development of learners’ writing ability. The role of teachers’ feedback and teachers’ and learners’ social interaction in the development of learners’ writing ability could potentially provide comprehensive, fair, reliable, and valid scores in this model.

Keywords


Article Title [Persian]

بررسی مقایسه ای آزمون نگارش در مدل آزمونی بر مبنای اکتیویتی تیوری و روش سنتی

Authors [Persian]

  • سید محمد علوی 1
  • شیوا کیوان پناه 2
  • فاطمه دانش 3
2 دانشگاه تهران
Abstract [Persian]

باور بر این است که سنجش  مهارت نگارش  بدلیل داوری  ذهنی وکیفی  و یا استفاده از سرفصل های نمره گذاری تحلیلی بطور بلقوه موجب سنجش بسیار موثقی نمی‌شود. به نظر می‌رسد که ارزیابی بر اساس یک مدل آزمونی بتواند این مشکلات را حل کند. برای دستیابی به این هدف، این مطالعه ابتدا یک مدل سنجشی بر مبنای نظریه فعالیت ایجاد کرده و سپس آنرا در سنجش عملکردهای نگارشی فراگیران ایرانی زبان انگلیسی بکار برده است. سپس، برای رسیدن به موثق بودن این مدل، نتایج آن را با نتایج دیدگاه سنتی مقایسه کرده است. یک گروه 29 نفری شامل فراگیران زبان انگلیسی که به مدت چهار هفته، هفته ای  یک نمونه نگارشی  تحویل دادند ، یک گروه شش نفری معلمهایی  که بازخوردی از اشکالات فراگیران در طول دوره داشتند و یک نمره کلی به نگارش آنها دادند ، و یک گروه دو نفری هم شامل ناظران این دوره که در پایان  دوره بر اساس سرفصل های  نمره گذاری تحلیلی  دوباره نمره گذاری کردند در این مطالعه  شرکت داشتند. نتایج نشان داد که بازخوردهای معلمها در هر نمونه نگارشی و ارتباط اجتماعی معلمها و فراگیران در بالابردن توانایی نگارش فراگیران  بسیار موثر بوده است  و بکارگیری این مدل در آزمون نگارش موجب متعهد شدن هم فراگیران و هم معلمها در بالابردن توانایی نگارش فراگیران شده و بطور بلقوه نمرات بسیار قابل قبول تر، عادلانه تر و قابل اعتمادتری به عملکرد نگارشی فراگیران داده است.

Keywords [Persian]

  • نظریه فعالیت
  • مدل آزمونی سنجشی بر مبنای نظریه فعالیت
  • سنجش مهارت نگارش
  • بازخورد معلم
Alderson, J. C. & Banerjee, J. (2001). 'Language testing and assessment (Part 1): state-of-the-art review'. Language Teaching, 34(4), 213 -36.
Barkaoui, K. (2010). Variability in ESL essay rating processes: The role of the rating scale and rater experience. Language Assessment Quarterly,7(1), 54-74.
Barnard Bachelor, R. (2017). Alternative Assessments and Student Perceptions in the World Language Classroom. Online Submission, 6, 30-44.
Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1, 1-47.
Chen, R., Sharman, R., Rao, H. R., & Upadhyaya, S. J. (2013). "Data Model Development for Fire Related.
Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Helsinki: Orienta-Konsultit.
Engeström, Y. (1990). When is a tool? Multiple meanings of artifacts in human activity. In Y. Engeström, Learning, working and imagining: Tweleve studies in activity theory. Helsinki: Orienta-Konsultit.
Engeström, Y. (1999) Innovative learning in work teams: analysing cycles of knowledge creation in practice, in: Y. ENGESTRÖM et al (Eds.) Perspectives on Activity Theory, (Cambridge, Cambridge University Press), 377-406.
González, E. F., Trejo, N. P. & Roux, R. (2017). Assessing EFL university students’ writing: a study of score reliability. Revista Electrónica de Investigación Educativa, 19(2), 91-103. https://doi.org/10.24320/redie.2017.19.2.928.
Hamayan, E.V. (1995). Approaches to alternative assessment. "Annual Review of Applied Linguistics," 15, 212-226.
Hattie, J. and Timperley. H. (2007). The Power of feedback. Review of Educational Research, 77, 81-112.
Jonassen, D. H., & Rohrer-Murphy, L. (1999). Activity theory as a framework for designing constructivist learning environments. Educational Technology Research and Development, 47, 61–79.
Karanasios, S., Allen, D., & Finnegan, P. (2015). "Information Systems Journal Special Issue On: Activity Theory in Information Systems Research," Information Systems Journal (25:3), pp. 309-313.
Kluger, A., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 2, 254 284. doi:10.1037/0033-2909.119.2.254
Lantolf, J. P., & Poehner, M. E. (2004). Dynamic assessment in the
language classroom (CALPER Professional Development Document CPDD-0411). University Park, PA: ThePennsylvania State University, Center for Advanced Language Proficiency Education andResearch.
Lepper, M. R., & Chabay, R. W. (1985). Intrinsic motivation and instruction: Conflicting views on the role of motivational processes in computer-based education. Educational Psychologist, 20(4), 217–230.
Lidz, Carol S. (1991). Practitioner's guide to dynamic assessment. New York, NY: Guilford Press.
Mattarima, K. & Hamdan, A. R. (2011). Understanding students learning strategies as an input context to design English classroom activities”. International Journal of Psychological Studies, 3 (2), 238-248.
Mory, E. H. (2004). Feedback research review. In D. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 745–783). Mahwah, NJ: Erlbaum Associates.
Mwanza, D. and Engestrom, Y. (2003). Pedagogical Adeptness in the Design of E-learning Environments: Experiences from the Lab@Future Project. In: Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2003, 7-11 Nov 2003, Phoenix. USA.
Narciss, S., & Huth, K. (2004). How to design informative tutoring feedback for multimedia learning. In H. M. Niegemann, D. Leutner, & R. Brunken (Ed.), Instructional design for multimedia learning (pp. 181-195). Munster, New York: Waxmann.
Nardi, B. (1996). Activity theory and human-computer interaction. In B. Nardi (Ed.), Contexts and consciousness. Activity theory and human-computer interaction (pp. 7-16). The MIT Press:Cambridge.
O’Farrell, C. (2016). Enhancing Student Learning through Assessment.
Pearson, P.C. (2004). Controversies in second language writing: Dilemmas and decisions in research and instruction. The University of Michigan Press.
Poehner,  M. E. (2007). Beyond the Test: L2 Dynamic Assessment and the Transcendence of Mediated Learning. Modern Language Journal, 91, 323-340.
Vygotsky, L. (1978). Mind in society: The development of higher psychological processes (Ed. By M. Cole, V. John-Steiner, S. Scribner, & E. Souberman). Cambridge, MA: Harvard University Press.
Wang, T. H. (2010). Web-based dynamic assessment: taking assessment as teaching and learning strategy for improving students’ e-Learning effectiveness. Computers & Education, 54(4), 1157–1166.
Wylie, C. & Lyon, C. (2016). Using the Formative Assessment Rubrics, Reflection and Observation Tools to Support Professional Reflection on Practice (Revised). Formative Assessment for Students and Teachers State Collaborative on Assessment and Student Standards of the Council of Chief State School Officers. Washington, DC.
Yamagata-Lynch, L. C. (2010). "Understanding Cultural Historical Activity Theory," in Activity Systems Analysis Methods. Springer, pp. 13-26.