Examining Cloze Tests as a Measure of Linguistic Complexity in L2 Writing

Chung, Eun Seon; Ahn, Soojin

doi:10.30961/lr.2019.55.3.627

Lang. Res. 55(3):627-649

pISSN: 0254-4474, eISSN: 2586-7113

DOI: https://doi.org/10.30961/lr.2019.55.3.627

Article

Examining Cloze Tests as a Measure of Linguistic Complexity in L2 Writing

Eun Seon Chung¹, Soojin Ahn²^,^†

Author Information & Copyright ▼

¹Hankuk University of Foreign Studies

²University of Seoul

^*Corresponding Author : Visiting Professor, Division of General English, University of Seoul, 163 Seoulsiripdaero, Dongdaemun-gu, Seoul 02504, Korea , E-mail: soojina2013@uos.ac.kr

ⓒ Copyright 2019 Language Education Institute, Seoul National University. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Oct 31, 2019 ; Revised: Dec 02, 2019 ; Accepted: Dec 18, 2019

Published Online: Dec 31, 2019

ABSTRACT

Given that cloze tests are strongly associated with learners’ writing proficiency, the present study examines the relationship between cloze test scores and specific linguistic features in second language (L2) writing. We investigate whether cloze tests can adequately and reliably measure linguistic features of syntactic and lexical complexity in L2 writing. Furthermore, the present study investigates the effect of the scoring method and L2 proficiency level on the relationship between cloze test scores and linguistic complexity features. Analysis of 60 students’ writing compositions found the syntactic and lexical complexity features in length-related measures, complex nominals, and lexical variation to be significantly correlated with their cloze test scores. There was no significant difference in the effects of the two scoring methods. Also, cloze tests were most informative for high-level learners and least informative for intermediate-level learners. Implications for the use of cloze tests as a diagnostic measure in L2 writing classrooms are provided.

Keywords: cloze test; syntactic complexity; lexical complexity; scoring method; L2 writing proficiency

References

Alderson, J. C. (1979a). The cloze procedure and proficiency in English as a foreign language. TESOL Quarterly, 13(2), 219-227 .

Alderson, J. C. (1979b). The effect on the cloze test of changes in deletion frequency. Journal of Research in Reading, 2(2), 108-119 .

Alderson, J. C. (1980). Native and nonnative speaker performance on cloze tests. Language Learning, 30(1), 59-76 .

Azizollah, D., Reza, Z., & Mohsen, R. (2012). Argumentative and narrative written task performance: Differential effects of critical thinking. International Journal of Research Studies in Language Learning, 24(1), 1-12 .

Bachman, L. F. (1985). Performance on cloze tests with fixed-ratio and rational deletions. TESOL Quarterly, 16, 61-70 .

Brown, J. D. (1980). Relative merits of four methods for scoring cloze tests. The Modern Language Journal, 64(3), 31-317 .

Brown, J. D. (1983). A closer look at cloze: Validity and reliability. In J. W. Oller, Jr. (Ed.), Issues in language testing research (pp. 237-250). Rowley, MA: Newbury House .

Brown, J. D., Yamashiro, A. D., & Ogane, E. (2001). The emperor's new cloze: Strategies for revising cloze tests. In T. Hudson & J. D. Brown (Eds.), A focus on language test development (pp.143-161). Honolulu, HI: University of Hawaii .

Chae, E., & Shin, J. (2015). A study of a timed cloze test for evaluating L2 proficiency. English Teaching, 70(3), 117-135 .

10.

Chapelle, C., & Abraham, R. G. (1990). Cloze method: What difference does it make? Language Testing, 7, 121-46 .

11.

Cobb, T. (2002). VocabProfilers [computer software]. Retrieved from https://www.lextutor.ca/vp/ .

12.

Cooper, T. C. (1976). Measuring written syntactic patterns of second language learners of German. Journal of Educational Research, 69(5), 176-183 .

13.

Crossley, S. A., & McNamara, D. S. (2009). Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing, 18(2), 119-135 .

14.

Crossley, S. A., Weston, J. L., Sullivan, S. T. M., & McNamara, D. S. (2011). The development of writing proficiency as a function of grade level: A linguistic analysis. Written Communication, 28(3), 282-311 .

15.

Crossley, S. A., Salsbury, T., & McNamara, D. S. (2012). Predicting the proficiency level of language learners using lexical indices. Language Testing, 29, 243-263 .

16.

Eckes, T., & Grotjahn, R. (2006). A closer look at the construct validity of C-tests. Language Testing, 23(3), 290-325 .

17.

Ferris, D. R. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28(2), 414-420 .

18.

Fotos, S. S. (1991). The cloze test as an integrative measure of EFL proficiency: A substitute for essays on college entrance examinations? Language Learning, 41(3), 313-336 .

19.

Gellert, A. S., & Elbro, C. (2013). Cloze tests may be quick, but are they dirty? Development and preliminary validation of a cloze test of reading comprehension. Journal of Psychoeducational Assessment, 31(1), 16-28 .

20.

Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123-145 .

21.

Guo, L., Crossley, S. A., & McNamara, D. S. (2013). Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing, 18, 218-238 .

22.

Hanania, E., & Shikhani, M. (1986). Interrelationships among three tests of language proficiency: Standardized ESL, cloze, and writing. TESOL Quarterly, 20(1), 97-109 .

23.

Harsch, C., & Hartig, J. (2016). Comparing C-tests and yes/no vocabulary size tests as predictors of receptive language skills. Language Testing, 33(4), 555-575 .

24.

Ionin, T., Montrul, S., & Crivos, M. (2013). A bidirectional study on the acquisition of plural noun phrase interpretation in English and Spanish. Applied Psycholinguistics, 34(3), 483-518 .

25.

Jarvis, S., Grant, L., Bikowski, D., & Ferris, D. (2003). Exploring multiple profiles of highly rated learner compositions. Journal of Second Language Writing, 12(4), 377-403 .

26.

Jung, YJ., Crossley, S., & McNamara, D. (2019). Predicting language writing proficiency in learner texts using computational tools. The Journal of Asia TEFL, 16(1), 37-52 .

27.

Kim, S.-S. (1994). The cloze procedure and EFL proficiency. English Teaching, 48, 127-150 .

28.

Kobayashi, M. (2002). Method effects on reading comprehension test performance: Text organization and response format. Language Testing, 19(2), 193-220 .

29.

Kyle, K., & Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly, 49(4), 757-786 .

30.

Lee, S. (1997). Cloze test as a measure of EFL writing proficiency. English Teaching, 52(3), 151-172 .

31.

Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4), 474-496 .

32.

Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college‐level ESL writers' language development. TESOL Quarterly, 45(1), 36-62 .

33.

Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners' oral narratives. The Modern Language Journal, 96(2), 190-208 .

34.

Markham, P. (1987). Rational deletion cloze processing strategies: ESL and native English. System, 15(3), 303-311 .

35.

McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010). Linguistic features of writing quality. Written Communication, 27(1), 57-86 .

36.

McNamara, D. S., Graesser, A.C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge: Cambridge University Press .

37.

Montrul, S. A. (1997). Transitivity alternations in second language acquisition: A crosslinguistic study of English, Spanish and Turkish. Unpublished doctoral dissertation. Montreal, Canada: McGill University .

38.

Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555-578 .

39.

Oller, W. (1972). Scoring methods and difficulty levels for cloze tests of ESL proficiency. The Modern Language Journal, 56(3), 151-158 .

40.

O'Neill, R., Cornelius, E. T., & Washburn, G. N. (1981). American kernel lessons: Advanced students' book. London: Longman .

41.

Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492-518 .

42.

Park, S-Y. (2012). A corpus-based study of syntactic complexity measures as development indices of college-level L2 learners' proficiency in writing. Korean Journal of Applied Linguistics, 28(3), 139-160 .

43.

Porter, D. (1978). Cloze procedure and equivalence. Language Learning, 28, 333-340 .

44.

Read, J. (2000). Assessing vocabulary. Oxford: Oxford University Press .

45.

Ryoo, Y. (2017). Predictability of the cloze test as a measure of written productive vocabulary. Modern English Education, 18(4), 25-45 .

46.

Schoonen, R., Gelderen, A. V., Stoel, R. D., Hulstijn, J., & Glopper, K. D. (2011). Modeling the development of L1 and EFL writing proficiency of secondary school students. Language Learning, 61(1), 33-79 .

47.

Shanahan, T., Kamil, M. L., & Tobin, A. W. (1982). Cloze as a measure of intersentential comprehension. Reading Research Quarterly, 17(2), 229-255 .

48.

Skehan, P. (2009). Modeling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30(4), 510-532 .

49.

Stubbs, J. B., & Tucker, G. R. (1974). The cloze test as a measure of English proficiency. The Modern Language Journal, 58(5/6), 239-241 .

50.

Taylor, W. L. (1953). Cloze procedure: A new tool for measuring readability. Journalism Quarterly, 30, 414-438 .

51.

Wolfe-Quintero, K., Inagaki, S., & Kim, H. Y. (1998). Second language development in writing: Measures of fluency, accuracy, and complexity. Honolulu, HI: University of Hawaii Press .