Your browser does not support the PubReader view.
Go to this page to see a list of supporting browsers.
The Korean Association for the Study of English Language and Linguistics
[ Article ]
Korea Journal of English Language and Linguistics - Vol. 23, No. 0, pp.1077-1093
ISSN: 1598-1398 (Print) 2586-7474 (Online)
Print publication date 30 Jan 2023
Received 18 Oct 2023 Revised 04 Nov 2023 Accepted 15 Nov 2023
DOI: https://doi.org/10.15738/kjell.23..202311.1077

대학생 영어 능력 평가를 위한 수준설정 사례 연구

이용상 ; 김은주
인하대학교
한양여자대학교
A case study on standard setting for English proficiency test for university students
Yongsang Lee ; Eun-Ju Kim
(First author) Associate Professor, Dept. of Education, Inha University, Tel: 032-860-7873 yong21c@gmail.com
(corresponding author) Associate Professor, Dept. of English, Hanyang Women’s University, Tel: 02-2290-2280 exk188@hywoman.ac.kr


© 2023 KASELL All rights reserved
This is an open-access article distributed under the terms of the Creative Commons License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This study applied a standard setting method for determining cutoff scores for university students’ English proficiency tests, which are criterion-referenced tests, conducted at universities and explored the application of this approach for enhancing the effectiveness of general English education at different levels. To differentiate students’ proficiency levels, the study used the cutoff scores determined by the Angoff method based on the English proficiency test items used for sub-divisions of general English courses at University A. University A conducts its sub-division classes in three proficiency levels, and, accordingly, this study employed level determination based on the three performance level descriptions(PLD). The results of the Angoff standard setting indicated that, as rounds of discussion progressed, the panel members tended to make decisions in favor of raising the cutoff scores, and the standard deviation of the cutoff scores estimated by the panel members tended to decrease demonstrating a narrowing of opinions among the panel members. However, it was confirmed that the estimation tendencies of specific panel members could influence the overall standard deviation, and, therefore, guidelines are needed to control such bias in future standard settings. The application of the cutoff scores determined through standard setting to assess students' English proficiency levels showed a difference in the distribution of level-based classes from what is currently being conducted at University A. This suggests that, for universities using fixed cutoff scores to divide classes into proficiency levels, the standard setting is necessary to ensure the appropriate division of level-based classes.

Keywords:

Standard setting, criterion-referenced test, university student English proficiency test, performance level description

References

  • 김성혜, 임자연(Kim, S. H. and J. Lim). 2013. 대학교양영어 프로그램의 운영현황(The current state of college English education in Korea). ≪현대영어교육≫(Mordern English Education) 14(2), 263-290.
  • 김성해, 최태환, 사재진, 홍서경 (Kim. S., T. Choi, J. Sa and S. Hong). 2021. 대학영어 단계별 교육과정에 대한 학습자 인식 및 만족도 연구: C대학 사례를 중심으로 (A study of learner perceptions and satisfactions of level-differentiated English education in college: Focused on the case of C university). ≪영어영문학≫(The Mirae Journal of English Language and Literature) 26(2), 155-176. [https://doi.org/10.46449/MJELL.2021.02.26.1.155]
  • 김성숙, 박은아, 서민희(Kim, S., E. Park and M. Seo). 2014. 고등학교 성취평가에서의 고정분할점수와 수준설정방법 적용 결과의 비교 분석(Adoptability of standard-setting methods to the high school level in the achievement standard assessment). ≪교육평가연구≫(Journal of Eucational Evaluation) 27(1), 1-22.
  • 김우형(Kim, W. H.). 2015. 수준별 교양영어교육을 위한 영어능력 평가진단 도구개발 연구(A study on the development of an English ability assessment tool for level-differentiated English education). ≪영어영문학연구≫(Studies in English Language & Literature) 41(1), 203-226. [https://doi.org/10.21559/aellk.2015.41.1.011]
  • 곽예린, 김경리(Gwak, Y and K. Kim). 2022. 중학교 수리적성 영역 검사에 대한 Modified-Angoff와 Bookmark 준거설정 방법 비교(A comparison of the modified-Angoff method and Bookmark method for standard setting on formative assessment). ≪교육과학연구≫(Journal of Educational Studies) 53(4), 59-82. [https://doi.org/10.15854/jes.2022.12.53.4.59]
  • 박연복, 이규민, 강상진(Park, Y., G. Lee and S. Kang). 2011. 군집분석을 이용한 수준설정 방법과 타당성 연구(A study of standard setting method using the cluster analysis and validity). ≪교육평가연구≫(Journal of Educational Evaluation) 24(3), 645-664.
  • 박인용, 송미영, 김성숙(Park, I., M. Song and S. Kim). 2014. 수준설정의 타당성 검증을 위한 평정자 기대 정답률의 예측오차 분석(Accuracy of estimating expected probability for MAP in standard setting). ≪교육과정평가연구≫(The Journal of Curriculum and Evaluation) 17(3), 223-245. [https://doi.org/10.29221/jce.2014.17.3.223]
  • 서정아(Seo, S). 2018. 대학 교양영어 수준별 수업에 대한 학생들의 인식에 관한 연구(A study of students’ perceptions towards level-differentiated general English classes in the university). ≪교육문화연구≫(Journal of Education & Culture) 24(5), 355-375. [https://doi.org/10.24159/joec.2018.24.5.355]
  • 서혜진(Seo, H.). 2020. 수준별 대학 교양영어에 대한 학습자 만족도 및 인식연구(A study of college students’ satisfaction and opinions on level-differentiated instruction of English in general education). ≪교사교육연구≫(Teacher Education Research) 59(4), 601-614.
  • 성태제, 시기자, 이경희, 박산하, 권승아(Seong, T., K. Si, K. Lee, S. Park and S. Kwon). 2016. 중학생용 CACV 종합진로직업적성검사 검사 매뉴얼(Middle School CACV comprehensive career aptitude test examination manual). 서울: 인싸이트(Seoul: Inpsyt).
  • 오은주(Oh, E.). 2022. 세계시민성 및 글로벌역량 함양을 위한 교양영어 수업운영 사례연구(Integrating global citizenship and global competence into a general English course: A case study). ≪국제이해교육연구≫(Journal of Education for International Understanding) 17(1), 93-156. [https://doi.org/10.35179/jeiu.2022.17.1.93]
  • 이영주, 박장희, 정준원, 김수정, 김윤덕(Lee, Y., J. Park, J. Jung, S. Kim and Y. Kim). 2017. 의과대학 수업수준에서의 준거설정 사례 연구(Study on standard setting methods applied to medical school class). ≪교육문화연구≫(Journal of Education & Culture) 23(2), 189-209. [https://doi.org/10.24159/joec.2017.23.2.189]
  • 이선경, 지은림(Lee, S and E. Ji). 2017. Rasch 방법과 Extended-Angoff 방법을 적용한 인성검사 준거설정 비교 연구(Comparing Rasch method and extended-Angoff method in standard setting for character Test). ≪교육평가연구≫(Journal of Educational Evaluation) 30(4), 761-789.
  • 임의진, 이용원, 전희성(Lim, E., Y. Lee and H. Jeon). 2019. 유럽 언어 공통기준 연계를 통한 영어 독해 시험의 수준설정: 수정된 앙고프 방법과 북마크 방법 비교(Standard setting to relate an English reading comprehension test to the CEFR: A comparison of modified Angoff and Bookmark methods). ≪교육평가연구≫(Journal of Educational Evaluation) 32(3), 523-548. [https://doi.org/10.31158/JEEV.2019.32.3.523]
  • 정양수(Jung, Y.). 2014. 교양영어 수준별 수업의 운영 효과 및 개선 방안에 관한 연구(A study on the implementation of level-specific general English and its effects). ≪언어학연구≫(Journal of Linguistic Studies) 19(3), 127-148.
  • Educational Testing Service. 2021. Mapping the Redesigned TOEIC Bridge Test Scores to Proficiency Levels of the Common European Framework of Reference for Languages. Princeton, NJ: Educational Testing Service.
  • Zieky, M. J., M. Perie and S. A. Livingston. 2008. Cutscores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.