Your browser does not support the PubReader view.
Go to this page to see a list of supporting browsers.
The Korean Association for the Study of English Language and Linguistics
[ Article ]
Korea Journal of English Language and Linguistics - Vol. 26, No. 0, pp.1098-1116
ISSN: 1598-1398 (Print) 2586-7474 (Online)
Print publication date 30 Sep 2026
Received 14 Jul 2026 Revised 01 Aug 2026 Accepted 18 Aug 2026
DOI: https://doi.org/10.15738/kjell.26..202609.1098

Calibrating Difficulty Across Lower-Level ELT Coursebooks: A Coh-Metrix Analysis of Texts by Major International Publishers

Andrew Garth
Assistant Professor, Department of English Dankook University 119 Dandae-ro, Anseo-dong, Cheonan-si, Chungnam, Korea, Tel: +82-41-550-3130 andy.garth@gmail.com


© 2026 KASELL All rights reserved
This is an open-access article distributed under the terms of the Creative Commons License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The current study investigated how textual difficulty progresses across internationally published English language teaching coursebooks designed for lower-level learners. In total, 1,154 reading passages from 60 coursebooks were examined at the beginner, elementary, and pre-intermediate levels, corresponding respectively to levels A1, A2, and B1 of the Common European Framework of Reference for Languages. Nine Coh-Metrix indices representing text length, syntactic complexity, lexical diversity, lexical sophistication, referential cohesion, connective cohesion, and semantic coherence were measured. Linear mixed-effects models were employed to examine differences between adjacent proficiency levels (i.e., A1-A2 and A2-B1) while accounting for variation across publishers, series, and textbooks. The results indicated that textual difficulty progressed selectively rather than uniformly across the lower levels of the Common European Framework of Reference for Languages. Text length, sentence length, and lexical diversity increased consistently across both transitions and accumulated to large differences from A1 to B1, whereas modifiers per noun phrase, referential cohesion, and semantic coherence did not change at any point. Furthermore, direct comparison of the two transitions found no significant difference in magnitude for any index, indicating that where difficulty increased, it increased by a comparable amount at each step.

Keywords:

Coh-Metrix, ELT reading passages, international ELT publishers, lower-level learners

Acknowledgments

The present research was supported by the research fund of Dankook University in 2025.

References

  • Byrnes, H. 1998. Reading in the beginning and intermediate college foreign language class. In G. S. Burkart, ed., Modules for the Professional Preparation of Teaching Assistants in Foreign Languages, 227-260. Washington, DC: Center for Applied Linguistics.
  • Cheng, Y.-S. and S.-C. Chang. 2022. Do senior high school English textbooks sufficiently prepare students for the high-stakes college entrance examinations? A corpus-based analysis of text difficulty. Journal of Textbook Research 15(1), 43-80.
  • Council of Europe. 2001. Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press.
  • Council of Europe. 2020. Common European Framework of Reference for Languages: Learning, Teaching, Assessment – Companion Volume. Strasbourg, France: Council of Europe Publishing.
  • Crossley, S., D. Allen and D. McNamara. 2011. Text readability and intuitive simplification: A comparison of readability formulas. Reading in a Foreign Language 23(1), 84-101. [https://doi.org/10.64152/10125/66657]
  • Crossley, S., J. Greenfield and D. McNamara. 2008. Assessing text readability using cognitively based indices. TESOL Quarterly 42(3), 475-493. [https://doi.org/10.1002/j.1545-7249.2008.tb00142.x]
  • Gelman, A. and H. Stern. 2006. The difference between “significant” and “not significant” is not itself statistically significant. The American Statistician 60(4), 328-331. [https://doi.org/10.1198/000313006X152649]
  • Grabe, W. 2009. Reading in a Second Language: Moving from Theory to Practice. New York: Cambridge University Press. [https://doi.org/10.1017/CBO9781139150484]
  • Grabe, W. and F. L. Stoller. 2020. Teaching and Researching Reading. 3rd ed. New York: Routledge. [https://doi.org/10.4324/9781315726274-2]
  • Graesser, A., D. McNamara, Z. Cai, M. Conley, H. Li and J. Pennebaker. 2014. Coh-Metrix measures text characteristics at multiple levels of language and discourse. The Elementary School Journal 115(2), 210-229. [https://doi.org/10.1086/678293]
  • Graesser, A., D. McNamara and J. Kulikowich. 2011. Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher 40(5), 223-234. [https://doi.org/10.3102/0013189X11413260]
  • Graesser, A., D. McNamara, M. Louwerse and Z. Cai. 2004. Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers 36(2), 193-202. [https://doi.org/10.3758/BF03195564]
  • Hanifah, M., D. Wahyuni and S. Haryati. 2022. The readability of reading texts in English textbook for Indonesian senior high school using Coh-Metrix. English Education 10(2), 81-92.
  • Krashen, S. 1982. Principles and Practice in Second Language Acquisition. Oxford: Pergamon Press.
  • Krashen, S. 1985. The Input Hypothesis: Issues and Implications. London: Longman.
  • Lee, S. 2013. A corpus-based analysis of the difficulty of the reading materials in high school English 1 and 2 textbooks using Coh-Metrix. Studies in Foreign Language Education 27(2), 131-148. [https://doi.org/10.16933/sfle.2013.27.2.131]
  • McNamara, D., A. Graesser, P. McCarthy and Z. Cai. 2014. Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge: Cambridge University Press. [https://doi.org/10.1017/CBO9780511894664]
  • Mishan, F. 2022. The global ELT coursebook: A case of Cinderella’s slipper? Language Teaching 55(4), 490-505. [https://doi.org/10.1017/S0261444820000646]
  • Nation, I. S. P. 2013. Learning Vocabulary in Another Language. 2nd ed. Cambridge: Cambridge University Press. [https://doi.org/10.1017/CBO9781139858656]
  • North, B. 2014. Putting the Common European Framework of Reference to good use. Language Teaching 47(2), 228-249. [https://doi.org/10.1017/S0261444811000206]
  • Plakans, L. and Z. Bilki. 2016. Cohesion features in ESL reading: Comparing beginning, intermediate and advanced textbooks. Reading in a Foreign Language 28(1), 79-100. [https://doi.org/10.64152/10125/66716]
  • Richards, J. 2001. Curriculum Development in Language Teaching. Cambridge: Cambridge University Press. [https://doi.org/10.1017/CBO9780511667220]
  • Ryu, J. and M. Jeon. 2020a. A Coh-Metrix analysis of the first grade middle school English textbooks revised by the 2015 National Curriculum. Korean Journal of English Language and Linguistics 20, 89-121.
  • Ryu, J. and M. Jeon. 2020b. An analysis of text difficulty across grades in Korean middle school English textbooks using Coh-Metrix. Journal of Asia TEFL 17(3), 921-936. [https://doi.org/10.18823/asiatefl.2020.17.3.11.921]
  • Tomlinson, B. 2012. Materials development for language learning and teaching. Language Teaching 45(2), 143-179. [https://doi.org/10.1017/S0261444811000528]
  • Tomlinson, B. and H. Masuhara. 2018. The Complete Guide to the Theory and Practice of Materials Development for Language Learning. Hoboken, NJ: Wiley Blackwell.
  • Vygotsky, L. 1978. Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press.
  • Westfall, J., D. Kenny and C. M. Judd. 2014. Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General 143(5), 2020-2045. [https://doi.org/10.1037/xge0000014]