Test Fairness in a Large-Scale High-Stakes Language Test

Abstract

Inquiry into fairness of a test has been recognized as an important research activity to direct efforts to reduce bias and discrimination against certain groups of test takers, create equal opportunities for test takers to demonstrate their knowledge and skills, and promote social justice. Given the importance of fairness in testing, my research examined the fairness of one large-scale high-stakes test in China—the Graduate School Entrance English Exam (GSEEE). To guide this doctoral research, I first drew on four sources of conceptual and empirical work to identify key issues encompassing test fairness. Informed by Willingham’s conceptual framework, I investigated the fairness of the GSEEE in two studies. In Study 1, I examined whether the GSEEE test items functioned differentially and brought potential bias towards test taker groups based on gender and academic background. In Study 2, I investigated perceptions of the fairness of the GSEEE as expressed by program administrators, teachers, and test takers. In conclusion, this research offers empirical information with regard to the fairness of the GSEEE from psychometric and stakeholder perspectives. The research also provides evidence that the conceptualization of test fairness is mediated by contextualized beliefs and traditions. Whether a test is perceived as fair or not is derived from considerations in both the testing process and the broad socio-cultural context.