Your browser does not support the PubReader view.
Go to this page to see a list of supporting browsers.
The Korean Association for the Study of English Language and Linguistics
[ Article ]
Korea Journal of English Language and Linguistics - Vol. 24, No. 0, pp.616-631
ISSN: 1598-1398 (Print) 2586-7474 (Online)
Print publication date 31 Jan 2024
Received 31 Mar 2024 Revised 18 Apr 2024 Accepted 02 May 2024
DOI: https://doi.org/10.15738/kjell.24..202407.616

A Corpus-Based Analysis of the Inter-Grade Continuity of the Reading Passages of High School Mock CSAT English Tests

Jisu Ryu ; Hyejin Lee ; Moongee Jeon
(first author) Konkuk University The Konkuk Research Institute for Multilingualism and Multiculturalism 120 Neungdong-ro, Gwangjin-gu, Seoul, Korea jsryu0508@konkuk.ac.kr
Wonkwang University Department of English Education 460 Iksan Daero, Iksan, Jeonbuk, Korea lee.hyejin0419@gmail.com
(corresponding author) Konkuk University Department of English Language & Literature 120 Neungdong-ro, Gwangjin-gu, Seoul, Korea mjeon1@konkuk.ac.kr


© 2024 KASELL All rights reserved
This is an open-access article distributed under the terms of the Creative Commons License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The aim of this study was to investigate the inter-grade continuity of reading passages of the high school mock College Scholastic Ability Test (CAST) English exams by using Coh-Metrix, a sophisticated text analysis tool. In the current study, a corpus consisting of 75 reading passage files, with 25 passage files for each grade level, was built based on the 2023 mock CSAT English exam administered by the Seoul Metropolitan Office of Education, which was then subjected to a wide range of Coh-Metrix indices covering various linguistic dimensions. These included basic counts, word frequencies, word features (imageability, concreteness, age of acquisition, familiarity), lexical diversity, personal pronouns, connectives, standard readability scores, syntactic complexity, co-reference cohesion, and semantic cohesion scores. The main findings from the broader literature on language learning and assessment indicated robust inter-grade continuity in the high school mock CSAT English test passages, particularly in terms of word count, average word and sentence length, word frequency, word concreteness, additive connectives, and standard readability indices. These findings suggest implications for improving the construction of reading passages in the high school mock CSAT English tests in terms of other less controlled features.

Keywords:

mock CAST English tests, continuity, corpus analysis, Coh-Metrix

Acknowledgments

This paper was supported by Konkuk University in 2022.

References

  • Ahn, H. and J. Bae. 2021. A study about the analysis of linguistic difficulty among English textbooks with 2015 Revised National Curriculum, EBS-CSAT prep books, and College Scholastic Ability Test. Secondary English Education 14(3), 39-58. [https://doi.org/10.20487/kasee.14.3.202108.39]
  • Chang, J. 2019. A comparison of lexical complexity in 2016-2019 CSAT English reading passages by the types of assessment (norm-referenced assessment vs. criterion-referenced assessment). The SNU Journal of Education Research 28(3), 85-110. [https://doi.org/10.54346/sjer.2019.28.3.85]
  • Chang, J. 2022. A multi-dimensional comparison of CSAT English reading passages before and after the use of criterion-referenced assessment. Language Research 58(1), 61-89. [https://doi.org/10.30961/lr.2021.58.1.61]
  • Cirilo, R. 1981. Referential coherence and text structure in story comprehension. Journal of Verbal Learning and Verbal Behavior 20, 358-367. [https://doi.org/10.1016/S0022-5371(81)90505-3]
  • Coltheart, M. 1981. The MRC psycholinguistic database. Quarterly Journal of Experimental Psychology 33, 497-505. [https://doi.org/10.1080/14640748108400805]
  • Doo, S. and I. Choi. 2020. A comparative analysis on difficulty of LC and RC of the KCSAT and the mock KCSAT English tests. English Language Assessment 15(2), 119-147. [https://doi.org/10.37244/ela.2020.15.2.119]
  • Graesser, A. C., M. Jeon, Z. Cai and D. S. McNamara. 2008. Automatic analyses of language, discourse, and situation models. In J. Auracher and W. van Peer, eds., New beginnings in literary studies, 72-88. Cambridge: Cambridge Scholars Publishing.
  • Graesser, A. C., M. Jeon, Y. Yan and Z. Cai. 2007. Discourse cohesion in text and tutorial dialogue. Information Design Journal 15(3), 199-213. [https://doi.org/10.1075/idj.15.3.02gra]
  • Graesser, A. C., D. S. McNamara, M. M. Louwerse and Z. Cai. 2004. Coh-Metrix: Analysis of text on cohesion and language. Behavioral Research Methods, Instruments, and Computers 36(2), 193-202. [https://doi.org/10.3758/BF03195564]
  • Han, J. and Y. Lee. 2022. A corpus-based analysis of the perceived difficulty increase in the 2022 CSAT English Test. Korean Journal of English Language and Linguistics 22, 336-354.
  • Jeon, M. 2015. An analysis of the continuity among middle school English textbooks with an automated language analysis program. Modern English Education 16(1), 195-218.
  • Kim, J. and I. Choi. 2015. A corpus-based comparative analysis of linguistic difficulty among high school English textbooks, EBS-CSAT prep books, and College Scholastic Ability test. Multimedia-Assisted Language Learning 18(1), 59-92. [https://doi.org/10.15702/mall.2015.18.1.59]
  • Kim, K. and K. Cha. 2021. An analysis on the vocabulary of recent CSAT’s English listening assessment: Lexical scales, word difficulty, and high frequency content words. English Language Teaching 33(2), 23-36.
  • Kim, S. 2021. A corpus-based analysis of linguistic difficulty of the CSAT English section under absolute grading. English Language Assessment 16(2), 205-227. [https://doi.org/10.37244/ela.2021.16.2.205]
  • Koh, N. and J. Shin. 2017. A comparison of the level of difficulty in the English reading part of the CSAT: Before and after the EBS-CSAT linkage policy. Secondary English Education 10(4), 3-24.
  • Landauer, T. K. 2007. LSA as a theory of meaning. In T. K. Landauer, D. S. McNamara and W. Kintsch, eds., Handbook of Latent Semantic Analysis, 3-34. Mahwah, NJ: Erlbaum. [https://doi.org/10.4324/9780203936399]
  • Lee, D. and M. Kang. 2015. The analysis of linkage of the Korea Educational Broadcasting System (EBS) with the College Scholastic Ability Test (CSAT) & trial test for CSAT and the study on improvements for the linkage(in the English Part of the Test). Secondary English Education 8(4), 69-92.
  • Lee, H. 2017. Analysis of the distribution of speech acts in the English conversations of the 2017 CSAT and CSAT simulations. Secondary English Education 10(3), 23-45.
  • Lee, H. 2018. Vocabulary analysis of the English section of the 2017 CSAT and CSAT simulations. International Language and Literature 41, 195-215.
  • Lee, H. 2020. Vocabulary analysis of CSAT English tests and CSAT-EBS preparation coursebooks, with reference to the reading tests. Modern English Education 21(3), 48-57. [https://doi.org/10.18095/meeso.2020.21.3.48]
  • Lee, M. 2021. The prediction of the English preliminary CSAT by high school English performance test and regular English exam. English Language Assessment 16(2), 187-204. [https://doi.org/10.37244/ela.2021.16.2.187]
  • Lu, X. 2010. Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics 15, 474-496. [https://doi.org/10.1075/ijcl.15.4.02lu]
  • Lu, X. 2012. The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal 96(2), 190-208. [https://doi.org/10.1111/j.1540-4781.2011.01232_1.x]
  • Millis, K. and M. Just. 1994. The influence of connectives on sentence comprehension. Journal of Memory and Language 33, 128-147. [https://doi.org/10.1006/jmla.1994.1007]
  • Murray, J. 1997. Connective and narrative text: The role of continuity. Memory & Cognition 25(2), 227-236. [https://doi.org/10.3758/BF03201114]
  • Nation, I. S. P. 2006. How large a vocabulary is needed for reading and listening? Canadian Modern Language Review 63, 59-82. [https://doi.org/10.3138/cmlr.63.1.59]
  • Oh, J. and Y. Shin. 2020. A corpus-based analysis of the linguistic complexity levels of reading passages in the Korean college entrance examination. English Language Teaching 32(2), 109-126.
  • Ryu, J. and M. Jeon. 2020. An analysis of the continuity of the listening scripts of middle school English textbooks with Coh-Metrix. English Language & Literature Teaching 26(3), 101-121. [https://doi.org/10.35828/etak.2020.26.3.101]
  • Ryu, J. and M. Jeon. 2021. An analysis of linguistic features in science textbooks across grade levels: Focus on text cohesion. Journal of the Korean Association for Science Education 41(2), 71-82.
  • Shin, D. 2020. A validity study on the item type of ‘identifying the emotional state or mood’ in the English section of CSAT based on sentiment analysis. Journal of the Korea English Education Society 19(4), 159-179.
  • Shin, Y. 2019. Analyzing CSAT reading passages by using Coh-Metrix and VocaProfile: Focusing on four years from 2016 to 2019. Journal of Language Sciences 26(4), 109-127. [https://doi.org/10.14384/kals.2019.26.4.109]
  • Song, J. and J. Kim. 2021. An analysis of the reading passages represented in the textbooks of middle/high school and national achievement English tests through syntactic complexity and readability. Secondary English Education 14(2), 48-66. [https://doi.org/10.20487/kasee.14.2.202105.48]
  • Yang, S. and D. Lee. 2019. A corpus-based analysis of the topic distribution and vocabulary level of textbooks, EBS materials, and CSATs. Journal of Learner-Centered Curriculum and Instruction 19(4), 711-729. [https://doi.org/10.22251/jlcci.2019.19.4.711]
  • Yun, J., M. Lee and Y. Park. 2012. The influence of the language of directions, questions, and choices in practice CSAT on learners’ test results. Korean Journal of Applied Linguistics 28(1), 59-85. [https://doi.org/10.17154/kjal.2012.03.28.1.59]