Your browser does not support the PubReader view.
Go to this page to see a list of supporting browsers.
The Korean Association for the Study of English Language and Linguistics
[ Article ]
Korea Journal of English Language and Linguistics - Vol. 24, No. 0, pp.735-753
ISSN: 1598-1398 (Print) 2586-7474 (Online)
Print publication date 31 Jan 2024
Received 04 Nov 2023 Revised 18 Jun 2024 Accepted 16 Jul 2024
DOI: https://doi.org/10.15738/kjell.24..202407.735

Exploring the Impact of Cohesion on L2 Writing Proficiency: Beyond the Influence of the Lexical and Syntactic Complexity and the Use of N-Grams

Xuemei Liang ; Shinwoong Lee
(First author) Lecturer, Department of English Yanbian University 977, Gongyuan Road, Yanji City, Jilin Province, PR China, Tel: 0086-433-273-2421 liangxuemei@ybu.edu.cn
(Corresponding author) Professor, Department of English Language and Literature Hanyang University 222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea, Tel: 02-2220-0745 shinwoonglee@hanyang.ac.kr


© 2024 KASELL All rights reserved
This is an open-access article distributed under the terms of the Creative Commons License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The current study aimed to investigate what cohesion indices could predict L2 writing proficiency and whether they could explain L2 writing proficiency significantly, even after controlling for the influence of the lexical and syntactic complexity and the use of n-grams. A Pearson correlation coefficient and a stepwise regression were conducted to examine the relationship between the use of cohesive devices and L2 writing proficiency by analyzing 1,200 essays written by Korean EFL university students. Four cohesion indices, including semantic relatedness and causal, temporal, and logical connectives, were found to be significant predictors of L2 writing proficiency. Furthermore, the results of hierarchical regression analysis revealed that even after controlling for the influence of lexical and syntactic complexity and the use of n-grams on the writing scores, the semantic relatedness was still significantly predictive of L2 writing proficiency, explaining 8% of the total variance of the writing scores. The findings suggested that semantic relatedness plays a crucial role in L2 writing proficiency, uniquely contributing to its development.

Keywords:

cohesion, semantic relatedness, lexical complexity, syntactic complexity, n-gram, L2 writing proficiency

Acknowledgments

This work was supported by the research fund of Hanyang University (HY-202100000003524).

References

  • Ädel, A. and B. Erman. 2012. Recurrent word combinations in academic writing by native and non-native speakers of English: A lexical bundles approach. English for Specific Purposes 31(2), 81-92. [https://doi.org/10.1016/j.esp.2011.08.004]
  • Ai, H. and X. Lu. 2010. A web-based system for automatic measurement of lexical complexity. In 27th Annual Symposium of the Computer-Assisted Language Consortium (CALICO-10), 8-2.
  • Ai, H. and X. Lu. 2013. A corpus-based comparison of syntactic complexity in NNS and NS university students’ writing. In A. Díaz-Negrillo, N. Ballier and P. Thompson, eds., Automatic Treatment and Analysis of Learner Corpus Data, 249-264. Amsterdam: John Benjamins. [https://doi.org/10.1075/scl.59.15ai]
  • Bestgen, Y. 2017. Beyond single-word measures: L2 writing assessment, lexical richness and formulaic competence. System 69, 65-78. [https://doi.org/10.1016/j.system.2017.08.004]
  • Bestgen, Y. and S. Granger. 2014. Quantifying the development of phraseological competence in L2 English writing: An automated approach. Journal of Second Language Writing 26, 28-41. [https://doi.org/10.1016/j.jslw.2014.09.004]
  • Biber, D. and B. Gray. 2016. Grammatical Complexity in Academic English: Linguistic Change in Writing. Cambridge: Cambridge University Press. [https://doi.org/10.1017/CBO9780511920776]
  • Chon, Y. V., D. Shin and G. E. Kim. 2021. Comparing L2 learners’ writing against parallel machine-translated texts: Raters’ assessment, linguistic complexity and errors. System 96, 102408. [https://doi.org/10.1016/j.system.2020.102408]
  • Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. London: Academic Press.
  • Cortes, V. 2013. Connecting lexical bundles and moves in research article introductions. Journal of English for Academic Purposes 12(1), 33-43. [https://doi.org/10.1016/j.jeap.2012.11.002]
  • Crossley, S. A. 2020. Linguistic features in writing quality and development: An overview. Journal of Writing Research 11(3), 415-443. [https://doi.org/10.17239/jowr-2020.11.03.01]
  • Crossley, S. A. and D. S. McNamara. 2009. Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing 18(2), 119-135. [https://doi.org/10.1016/j.jslw.2009.02.002]
  • Crossley, S. A. and D. S. McNamara. 2012a. Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading 35(2), 115-135. [https://doi.org/10.1111/j.1467-9817.2010.01449.x]
  • Crossley, S. A. and D. S. McNamara. 2012b. Detecting the first language of second language writers using automated indices of cohesion, lexical sophistication, syntactic complexity and conceptual knowledge. In S. Jarvis and S. A. Crossley, eds., Approaching Language Transfer through Text Classification: Explorations in the Detection-based Approach, 106-126. Bristol: Multilingual Matters. [https://doi.org/10.21832/9781847696991-005]
  • Crossley, S. A. and D. S. McNamara. 2013. Applications of text analysis tools for spoken response grading. Language Learning & Technology 17, 171-192.
  • Crossley, S. A., K. Kyle and D. S. McNamara. 2016. The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. Journal of Second Language Writing 32, 1-16. [https://doi.org/10.1016/j.jslw.2016.01.003]
  • Crossley, S. A., K. Kyle, L. K. Allen, L. Guo and D. S. McNamara. 2014. Linguistic microfeatures to predict L2 writing proficiency: A case study in automated writing evaluation. Journal of Writing Assessment 7(1), 1-34.
  • Davies, M. 2009. The 385+ million word Corpus of Contemporary American English (1990-2008+): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics 14, 159-190. [https://doi.org/10.1075/ijcl.14.2.02dav]
  • Douglas, S. R. 2013. The lexical breadth of undergraduate novice level writing competency. The Canadian Journal of Applied Linguistics 16(1), 152-170.
  • Eddington, D. 2015. Statistics for Linguistics Research. Cambridge Scholars Publishing.
  • Garner, J., S. A. Crossley and K. Kyle. 2019. N-gram measures and L2 writing proficiency. System 80, 176-187. [https://doi.org/10.1016/j.system.2018.12.001]
  • Graesser, A. C., D. S. McNamara, M. M. Louwerse and Z. Cai. 2004. Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods 36(2), 193-202. [https://doi.org/10.3758/BF03195564]
  • Guo, L., S. A. Crossley and D. S. McNamara. 2013. Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing 18(3), 218-238. [https://doi.org/10.1016/j.asw.2013.05.002]
  • Jung, Y., S. A. Crossley and D. S. McNamara. 2019. Predicting second language writing proficiency in learner texts using computational tools. Journal of Asia TEFL 16(1), 37-52. [https://doi.org/10.18823/asiatefl.2019.16.1.3.37]
  • Kelly, P. C. 1988. Accessing a Global Language: A Comparison of G-TELP and TOEFL Performance. Doctoral dissertation, University of San Diego, San Diego, CA, USA.
  • Kim, H. and H. Seol. 2021. A study on the validity of the G-TELP level test. The Research Institute of Korean Education 39, 23-38. [https://doi.org/10.22327/kei.2021.39.4.023]
  • Kim, J. 2014. Predicting L2 writing proficiency using linguistic complexity measures: A corpus-based study. Korea Association of Teachers of English 69(4), 27-51. [https://doi.org/10.15858/engtea.69.4.201412.27]
  • Kim, J. 2022. The use of cohesive devices in Korean EFL writing across different proficiency levels. Korean Journal of English Language and Linguistics 22, 1078-1100.
  • Kim, M. and S. A. Crossley. 2018. Modeling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing. Assessing Writing 37, 39-56. [https://doi.org/10.1016/j.asw.2018.03.002]
  • Kim, S. and M. Kessler. 2022. Examining L2 English university students’ uses of lexical bundles and their relationship to writing quality. Assessing Writing 51, 100589. [https://doi.org/10.1016/j.asw.2021.100589]
  • Kyle, K. and S. A. Crossley. 2015. Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly 49(4), 757-786. [https://doi.org/10.1002/tesq.194]
  • Kyle, K. and S. A. Crossley. 2016. The relationship between lexical sophistication and independent and source-based writing. Journal of Second Language Writing 34, 12-24. [https://doi.org/10.1016/j.jslw.2016.10.003]
  • Kyle, K., S. A. Crossley and C. Berger. 2018. The tool for the automatic analysis of lexical sophistication (TAALES): version 2.0. Behavior Research Methods 50(3), 1030-1046. [https://doi.org/10.3758/s13428-017-0924-4]
  • Lee, C., H. Ge and E. Chung. 2021. What linguistic features distinguish and predict L2 writing quality? A study of examination scripts written by adolescent Chinese learners of English in Hong Kong. System 97, 102461. [https://doi.org/10.1016/j.system.2021.102461]
  • Lee, Y. 2021. The relationship between text cohesion features and English proficiency for Korean college students. Korean Journal of English Language and Linguistics 21, 435-449
  • Lu, X. 2010. Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics 15(4), 474-496. [https://doi.org/10.1075/ijcl.15.4.02lu]
  • Lu, X. 2011. A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly 45(1), 36-62. [https://doi.org/10.5054/tq.2011.240859]
  • Lu, X. 2012. The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal 96(2), 190-208. [https://doi.org/10.1111/j.1540-4781.2011.01232_1.x]
  • McNamara, D. S., S. A. Crossley and P. M. McCarthy. 2010. Linguistic features of writing quality. Written Communication 27(1), 57-86. [https://doi.org/10.1177/0741088309351547]
  • McNamara, D. S., A. C. Graesser, P. M. McCarthy and Z. Cai. 2014. Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge: Cambridge University Press. [https://doi.org/10.1017/CBO9780511894664]
  • Nasseri, M. and P. Thompson. 2021. Lexical density and diversity in dissertation abstracts: Revisiting English L1 vs. L2 text differences. Assessing Writing 47, 100511. [https://doi.org/10.1016/j.asw.2020.100511]
  • O’Leary, J. A. and R. Steinkrauss. 2022. Syntactic and lexical complexity in L2 English academic writing: Development and competition. Ampersand 9, 100096. [https://doi.org/10.1016/j.amper.2022.100096]
  • Read, J. 2000. Assessing Vocabulary. Cambridge: Cambridge University Press. [https://doi.org/10.1017/CBO9780511732942]
  • Ryu, J. 2020. Predicting second language writing proficiency in the different genres of writing using computational tools. Korean Journal of Applied Linguistics 36(1), 141-170. [https://doi.org/10.17154/kjal.2020.3.36.1.141]
  • Salazar, D. 2014. Lexical Bundles in Native and Non-Native Scientific Writing: Applying a Corpus-Based Study to Language Teaching. Amsterdam: John Benjamins. [https://doi.org/10.1075/scl.65]
  • Stubbs, M. 2007. An example of frequent English phraseology: Distribution, structures and functions. In R. Facchinetti, ed., Corpus Linguistics 25 Years On, 87-105. Leiden: Brill. [https://doi.org/10.1163/9789401204347_007]
  • Tywoniw, R. and S. A. Crossley. 2019. The effect of cohesive features in integrated and independent L2 writing quality and text classification. Language Education and Assessment 2(3), 110-134. [https://doi.org/10.29140/lea.v2n3.151]
  • Utsunomiya, Y., M. Maruyama and S. Ogasawara. 2016. Estimating TOEIC scores using G-TELP scores: A Bayesian model in a Japanese national university. JLTA Journal 19, 27-45. [https://doi.org/10.20622/jltajournal.19.0_27]
  • Yang, W. and Y. Sun. 2012. The use of cohesive devices in argumentative writing by Chinese EFL learners at different proficiency levels. Linguistics and Education 23(1), 31-48. [https://doi.org/10.1016/j.linged.2011.09.004]
  • Yoon, H. J. 2021. Challenging the connection between task perceptions and language use in L2 writing: Genre, cognitive task complexity, and linguistic complexity. Journal of Second Language Writing 54, 100857. [https://doi.org/10.1016/j.jslw.2021.100857]
  • Zhang, X., X. Lu and W. Li. 2022. Beyond differences: Assessing effects of shared linguistic features on L2 writing quality of two genres. Applied Linguistics 43(1), 168-195. [https://doi.org/10.1093/applin/amab007]
  • Zhang, X. and W. Li. 2021. Effects of n-grams on the rated L2 writing quality of expository essays: A conceptual replication and extension. System 97, 102437. [https://doi.org/10.1016/j.system.2020.102437]