Exploring the Effects of Chatbot-Assisted Language Learning on Learners’ Language Achievement and Sense of Metavolvement

Document Type : Research Paper

Authors

1 Ph.D. Candidate, Department of English, Ferdowsi University of Mashhad, Iran

2 Professor, Department of English, Ferdowsi University of Mashhad, Iran

3 Assistant Professor, Department of English, Ferdowsi University of Mashhad, Iran

Abstract

Artificial intelligence (AI) is changing language education through personalized and interactive practice. However, its impact on language achievement and emotional aspects of learning is unclear. Therefore, this mixed-methods study investigated the effects of Chatbot-Assisted Language Learning (CHALL) on learners’ English language achievement and sense of metavolvement—the deepest level of metacognitive engagement. In phase 1, the Sense of Metavolvement Scale (SOMS) was developed and validated. In phase 2, following a quasi-experimental design, 44 intermediate Iranian EFL learners were divided into an experimental group (n=22, using the AI chatbot Pi) and a control group (n=22, receiving conventional classroom instruction). Both groups had pre- and post-tests for language achievement and metavolvement, and the experimental group sat for semi-structured interviews post-intervention. There were significant gains in the experimental group’s in-class metavolvement and language achievement. There were also significant differences between the two groups in post-test metavolvement, while there was no significant difference in their post-test language achievement. Furthermore, interview data illustrated CHALL as a low-anxiety practice context with cultural and pragmatic limitations. Therefore, CHALL can complement but not replace conventional instruction.

Keywords

Main Subjects

References

Armstrong, D., Gosling, A., Weinman, J., & Marteau, T. (1997). The place of inter-rater reliability in qualitative research: An empirical study. Sociology, 31(3), 597–606.
Belpaeme, T., Vogt, P., Van den Berghe, R., Bergmann, K., Göksun, T., De Haas, M., ... & Pandey, A. K. (2018). Guidelines for designing social robots as second language tutors. International Journal of Social Robotics, 10(3), 325–341. https://doi.org/10.1007/s12369-018-0477-0
Bernstein, M. J., Sacco, D. F., Yong, S. G., Hugenberg, K., & Cook, E. (2010). Being "in" with the in-crowd: The effects of social exclusion and inclusion are enhanced by the perceived essentialism of ingroups and outgroups. Personality and Social Psychology Bulletin, 36(8), 999–1009. https://doi.org/10.1177/0146167210376059
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Sage.
Chapelle, C. (2001). Computer applications in second language acquisition: Foundations for teaching, testing and research. Cambridge University Press.
Duong, T. & Suppasetseree, S. (2024). the effects of an artificial intelligence voice chatbot on improving vietnamese undergraduate students’ English speaking skills. International Journal of Learning Teaching and Educational Research, 23(3), 293-321. https://doi.org/10.26803/ijlter.23.3.1
Fryer, L., Coniam, D., Carpenter, R., & Lăpușneanu, D. (2020). Bots for language learning now: Current and future directions. Language Learning & Technology, 24(2), 8–22.
Gholami, M. J. and Al Abdwani, T. (2024). The Rise of Thinking Machines: A Review of Artificial Intelligence in Contemporary Communication. Journal of Business, Communication & Technology, 3(1), 29-43. https://doi.org/10.56632/bct.2024.3103
Golonka, E. M., Bowles, A. R., Frank, V., Richardson, D. L., & Freynik, S. (2014). Technologies for foreign language learning: A review of technology types and their effectiveness. Computer Assisted Language Learning, 27(1), 70–105. https://doi.org/10.1080/09588221.2012.700315
Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot‐supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12505
Kim, H. S., Kim, N. Y., & Cha, Y. (2021). Is it beneficial to use AI chatbots to improve learners' speaking performance? Journal of Asia TEFL, 18(1), 161–178.
Krashen, S. D. (1989). We acquire vocabulary and spelling by reading: Additional evidence for the input hypothesis. Modern Language Journal, 73(4), 440–463. https://doi.org/10.1111/j.1540-4781.1989.tb05325.x
Latham-Koenig, C., Oxenden, C., & Lambert, J. (2020). American English File 3 (2nd ed.). Oxford University Press.
Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. C. Ritchie & T. K. Bhatia (Eds.), Handbook of language acquisition: Vol. 2. Second language acquisition (pp. 413–468). Academic Press.
Lyster, R., & Saito, K. (2010). Oral feedback in classroom SLA: A meta-analysis. Studies in Second Language Acquisition, 32(2), 265–302. https://doi.org/10.1017/S0272263109990520
Nagata, R., Hashiguchi, T., & Sadoun, D. (2020). Is the simplest chatbot effective in English writing learning assistance? In M. D. Lytras, P. Ordoñez de Pablos, & M. Helfert (Eds.), Computational linguistics: 16th International Conference of the Pacific Association for Computational Linguistics, PACLING 2019, Hanoi, Vietnam, October 11–13, 2019, Revised Selected Papers (pp. 245–256). Springer Singapore.
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1–13.
Panda, S., & Kaur, N. (2023). Exploring the viability of ChatGPT as an alternative to traditional chatbot systems in library and information centers. Library Hi Tech News, 40(3), 22–25.
Patel, R., Al-Hassan, S., & Kim, Y. (2023). Cultural responsiveness in AI language tools: A critical review. ReCALL, 35(2), 210–225. https://doi.org/10.1017/S0958344023000079
Pishghadam, R. (2016). Introducing emotioncy tension as a potential source of identity crisis. Paper presented at the 16th Interdisciplinary Conference on Cultural Identity and Philosophy of the Self, London, England.
Pishghadam, R., & Shayesteh, S. (2017). Emo-sensory expression at the crossroads of emotion, sense, and language: A case of color-emotion associations. International Journal of Society, Culture, & Language (IJSCL), 5(2), 15–25.
Pishghadam, R., Al Abdwani, T., Jajarmi, H., & Shayesteh, S. (2023a). Enhancing general and language aptitude tests by incorporating cultural and emo-sensory constructs. International Journal of Society, Culture and Language, 11(3), 1–12. https://doi.org/10.22034/ijscl.2023.704891
Pishghadam, R., Baghaei, P., & Seyednozadi, Z. (2017). Introducing emotioncy as a potential source of test bias: A mixed Rasch modeling study. International Journal of Testing, 17(2), 127–140. https://doi.org/10.1080/15305058.2016.1183208
Pishghadam, R., Ebrahimi, S., & Tabatabaiean, M. (2019a). A Novel Approach to Psychology of Language Education. FUM Press.
Pishghadam, R., Ebrahimi, S., Rajabi Esterabadi, A., & Parsae, A. (2023b). Emotions and success in education: From apathy to transpathy. Journal of Cognition, Emotion & Education, 1(1), 1–16. https://doi.org/10.22034/cee.2023.172495
Pishghadam, R., Shakeebaee, G., & Rahmani, S. (2019b). Sensory capital in education: The missing piece? Italian Journal of Sociology of Education, 11, 1–30. https://doi.org/10.14658/pupj-ijse-2019-3-15
Qasem, F., Ghaleb, M., Mahdi, H. S., Al Khateeb, A., & Al Fadda, H. (2023). Dialog chatbot as an interactive online tool in enhancing ESP vocabulary learning. Saudi Journal of Language Studies, 3(2), 76–86.
Stockwell, G. (2010). Using mobile phones for vocabulary activities: Examining the effect of the platform. Language Learning & Technology, 14(2), 95–110.
Ullman, J. B. (2001). Structural equation modeling. In B. G. Tabachnick & L. S. Fidell (Eds.), Using multivariate statistics (4th ed., pp. 653–771). Allyn & Bacon.
Viberg, O., & Grönlund, Å. (2013). Cross-cultural analysis of users' attitudes toward the use of mobile devices in second and foreign language learning in higher education: A case from Sweden and China. Computers & Education, 69, 169–180. https://doi.org/10.1016/j.compedu.2013.07.014
Warschauer, M., & Healey, D. (1998). Computers and language learning: An overview. Language Teaching, 31(2), 57–71. https://doi.org/10.1017/S0261444800012970
Wei, L. (2023). Artificial intelligence in language instruction: Impact on English learning achievement, L2 motivation, and self-regulated learning. Frontiers in Psychology, 14, 1261955. https://doi.org/10.3389/fpsyg.2023.1261955
Wu, X., & Li, R. (2024). Unraveling effects of AI chatbots on EFL learners’ language skill development: A meta-analysis. The Asia-Pacific Education Researcher. 33(3), 1-15. https://doi.org/10.1007/s40299-024-00853-2
Yang, H., & Kim, H. (2021). Development and application of AI chatbot for cabin crews. Korea Journal of English Language and Linguistics, 21(0), 1085–1104. http://journal.kasell.or.kr
Yang, H., Kim, H., Lee, J. H., & Shin, D. (2022). Implementation of an AI chatbot as an English conversation partner in EFL speaking classes. ReCALL, 34(3), 327–343. https://doi.org/10.1017/S095834402200007X
Zhai, C., & Wibowo, S. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers & Education: Artificial Intelligence, 4, Article 100134. https://doi.org/10.1016/j.caeai.2023.100134
Zhang, R., Zou, D., & Cheng, G. (2023). A review of chatbot-assisted learning: Pedagogical approaches, implementations, factors leading to effectiveness, theories, and future directions. Interactive Learning Environments, 1–29. https://doi.org/10.1080/10494820.2023.2207430

Files

Share

How to cite

Statistics