Effects of ChatGPT on Korean EFL Learners’ Main-Idea Reading Comprehension via Top-Down Processing

Kim, Rakhun

doi:10.30961/lr.2024.60.1.83

Lang. Res. 60(1):83-106

pISSN: 0254-4474, eISSN: 2586-7113

DOI: https://doi.org/10.30961/lr.2024.60.1.83

Article

Effects of ChatGPT on Korean EFL Learners’ Main-Idea Reading Comprehension via Top-Down Processing

Rakhun Kim ¹ ^, ^†

Author Information & Copyright ▼

¹Hankuk University of Foreign Studies

^†Corresponding Author: Adjunct Professor Department of English Education Hankuk University of Foreign Studies 107, Imun-ro, Dongdaemun-gu, Seoul 02450, Korea E-mail: kimrhee22@hufs.ac.kr

ⓒ Copyright 2024 Language Education Institute, Seoul National University. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Nov 04, 2023 ; Revised: Apr 15, 2024 ; Accepted: Apr 24, 2024

Published Online: Apr 30, 2024

ABSTRACT

This study quantitatively and qualitatively investigated the effects of ChatGPT on top-down processing in English reading comprehension among Korean English as a foreign language (EFL) learners. Participants were divided into experimental and control groups of 20 individuals each, completed pretest, immediate posttest, and delayed posttest sessions, and received explicit main idea identification instructions for Finding the Main Idea (FMI) test items from the Korean College Scholastic Aptitude Test. However, only the experimental group had access to ChatGPT. The results showed a statistically significant improvement in top-down processing skills in the experimental group. Additionally, a qualitative analysis of interviews with learners and ChatGPT-human interaction data revealed responses to ChatGPT’s use in English reading comprehension, emphasizing its potential benefits and challenges. These findings highlight the significance of balancing technological integration with pedagogical instruction to optimize learning experiences in the Korean EFL context.

Keywords: ChatGPT; artificial intelligence; top-down processing; reading comprehension; Korean EFL learners

References

Alkaissi, H., & McFarlane, S. I. (2023). Artificial hallucinations in ChatGPT: Implications in scientific writing. Cureus, 15(2).

Bang, J. (2020). The comparison of metacognitive awareness of reading strategy use among gifted learners in science, mathematics, and general middle school learners during english reading. Studies in English Language & Literature, 46(1), 125-145.

Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., V. Do, Q., Xu, Y., & Fung, P. (2023). A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023.

Bengio, Y., Ducharme, R., Vincent, P., & Jauvin, C. (2003). A neural probabilistic language model. Journal of Machine Learning Research, 3, 1137-1155.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.

Canale, M., & Swain, M. (1990). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1, 1-47.

Carrell, P. L. (1998). Can reading strategies be successfully taught?. Australian Review of Applied Linguistics, 21(1), 1-20.

Chamot, A. U., & O'malley, J. M. (1994). The CALLA handbook: Implementing the cognitive academic language learning approach. New York: Longman.

Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.

10.

Chong, I., & Kim, H. (2023). Effects of explicit reading strategy instruction on high and low English proficiency learners. Journal of Learner-Centered Curriculum and Instruction, 23(4), 715-731.

11.

Dechant, E. (1991). Understanding and teaching reading: An interactive model. New Jersey: Lawrence Erlbaum.

12.

Gers, F. A., Schmidhuber, J., & Cummins, F. (2000). Learning to forget: Continual prediction with LSTM. Neural Computation, 12(10), 2451-2471.

13.

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. Cambridge: MIT Press.

14.

Goodman, K. (1982). Language and literacy. Boston: Routledge & Kegan Paul.

15.

Goodman, K. (1998). The reading process. In P. Carrell, J. Devine, & D. Eskey (Eds), Interactive approaches to second language reading (pp. 11-21). Cambridge: Cambridge University Press.

16.

Grabe, W. (1998). Reassessing the term interactive. In P. Carrell, J. Devine & D. Eskey, (Eds.), Interactive approaches to second language reading (pp. 56-70). Cambridge: Cambridge University Press.

17.

Halliday, M. A., & Hasan. R. (1976). Cohesion in English. London: Longman.

18.

Hildyard, A., & Olson, D. R. (1978). Memory and inference in the comprehension of oral and written discourse. Discourse Processes, 1(2), 91-117.

19.

Hedge, T. (2000). Teaching and learning in the language classroom. Oxford: Oxford University Press.

20.

Hong-Nam, K., & Page, L. (2014). Investigating metacognitive awareness and reading strategy use of EFL Korean university students. Reading Psychology, 35(3), 195-220.

21.

Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 1-20.

22.

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., Weller, J., Kuhn, J., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274.

23.

Kim, R. (2023a). Effects of ChatGPT on the cognitive processing of K-CSAT English reading tasks by Korean high school learners: A preliminary study. Secondary English Education, 16(2), 179-205.

24.

Kim, T. (2023b). A field study on the development of teaching materials for secondary English and its use of ChatGPT. Secondary English Education, 16(2), 207-218.

25.

Kintsch, W. (1979). On modeling comprehension. Educational Psychologist, 14(1), 3-14.

26.

Kintsch, W., & Van Dijk, T. A. (1978). Toward a model of text comprehension and production. Psychological Review, 85(5), 363-394.

27.

Koda, K. (2007). Reading and language learning: Crosslinguistic constraints on second language reading development. Language Learning, 57(1), 1-44.

28.

Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537-550.

29.

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90.

30.

Kwon, S. K., & Lee, Y. (2023). Investigating the performance of generative AI ChatGPT's reading comprehension ability. Journal of the Korea English Education Society, 22(2), 147-172.

31.

Liu, G., & Ma, C. (2023). Measuring EFL learners' use of ChatGPT in informal digital learning of English based on the technology acceptance model. Innovation in Language Learning and Teaching, 1-14.

32.

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410.

33.

Maria, K., & MacGinitie, W. H. (1982). Reading comprehension disabilities: Knowledge structures and non-accommodating text processing strategies. Annals of Dyslexia, 32(1), 31-59.

34.

Marslen-Wilson, W. D., & Welsh, A. (1978). Processing interactions and lexical access during word recognition in continuous speech. Cognitive Psychology, 10(1), 29-63.

35.

Merriam, S. B., & Tisdell. E. J. (2015). Qualitative research: A guide to design and implementation (4th ed.). Jossey-Bass.

36.

Meyer, B. J. F., & Freedle, R. O. (1984). Effects of discourse type on recall. American Educational Research Journal, 21, 121-143.

37.

Nuttall, C. (2000). Teaching reading skills in a foreign language. London: Heinemann.

38.

Palinscar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1(2), 117-175.

39.

Park, J. (2018). The impact of EFL students' English proficiency on reading strategies and usage of annotation tools. English Language Assessment, 13, 85-120.

40.

R Development Core Team. (2008). R: A language and environment for statistical computing [Computer software]. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.Rproject.org.

41.

Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9), 5783.

42.

Reinders, H., & Benson, P. (2017). Research agenda: Language learning beyond the classroom. Language Teaching, 50(4), 561-578.

43.

Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?. Journal of Applied Learning and Teaching, 6(1), 1-22.

44.

Rumelhart, D. E. (1977). Toward an interactive model of reading. In S. Dornic (Ed.). Attention and performance (Vol. 6). Hillsdale: Erlbaum.

45.

Seok, W. (2024). Korean secondary English teachers' use of ChatGPT as teaching and learning tool: Practices and perceptions [Unpublished MS dissertation]. Korea National University of Education.

46.

Shaikh, S., Yayilgan, S. Y., Klimova, B., & Pikhart, M. (2023). Assessing the usability of ChatGPT for formal English language learning. European Journal of Investigation in Health, Psychology and Education, 13(9), 1937-1960.

47.

Shin, D. (2023a). A case study on english test item development training for secondary school teachers using AI tools: Focusing on ChatGPT. Language Research, 59(1), 21-42.

48.

Shin, D. (2023b). Utilizing ChatGPT in guided writing activities. Journal of the Korea English Education Society, 22(2), 197-217.

49.

Shin, D., Jung, H., & Lee, Y. (2023). Exploring the potential of using ChatGPT as a content-based English learning and teaching tool. Journal of the Korea English Education Society, 22(1), 171-192.

50.

Sousa, D. A. (2005). How the brain learns (3rd ed.). Thousand Oaks, CA: Corwin Press/Sage Publications.

51.

Stauffer, R. G. (1969). Directing reading maturity as a cognitive process. New York: Harper & Row.

52.

Szabo, A. (2023). ChatGPT is a breakthrough in science and education but fails a test in sports and exercise psychology. Baltic Journal of Sport and Health Sciences, 1(128), 25-40.

53.

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15.

54.

Urquhart, A. H. & Weir, C. (1998). Reading in a second language: Process, product and practice. London: Longman.

55.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, N., Jones, L., Gomez, N., A., Kaiser, L., & Polosukhin. I., (2017). Attention is all you need. In Advances in neural information processing systems, 5998-6008.

56.

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., & Zhou, D. (2022). Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903.

57.

Yang, H., & Jeong, Y. (2008). Understanding educational grammar. Seoul: Hankook Publishing.

58.

Zhang, C., Zhang, C., Li, C., Qiao, Y., Zheng, S., Kumar, S. D., Zhang, M., Uk Kim, J., Kim, S. T., Choi, J., Park, G. M., Bae, S. H., Lee, L. H., Hui, P., Kweon, I. S., & Hong, C. (2023). One small step for generative AI, one giant leap for AGI: A complete survey on ChatGPT in AIGC era. arXiv preprint arXiv:2304.0648.