Stakeholder voices: Validity argument for score meaning and use of the Test of English for International Communication (TOEIC) in international business workplaces in South Korea

Abstract

Competent English communication skills are essential to success within international business workplaces. To assess employees’ qualifications for international businesses, international companies have used the Test of English for International Communication (TOEIC) over the past 30 years in South Korea, which houses the largest TOEIC test taker population worldwide. However, some researchers argue that the TOEIC, designed by Educational Testing Service (ETS), may not fully reflect the use of English in international business contexts. In such contexts, the focus is interactive communication skills, with cognizance of different English accents and business partners’ cultures. Limited empirical studies have investigated the extent to which TOEIC scores represent English constructs and how much the TOEIC achieves its designated purposes in international business workplaces. Drawing on an argument-based validation framework using the stakeholder approach, the perceived constructs of English in international business workplaces and the extent to which the TOEIC’s designated purposes were achieved were investigated by collecting various validity-related evidence from multiple stakeholders, i.e., ETS; YBM, the TOEIC administrator; and employers and employees in international business workplaces. Multiple methods (i.e., documents from ETS and YBM, interviews with six employers and ten employees, and online questionnaires with 281 employees in international business companies in Korea) were used to juxtapose the findings from each stakeholder group. The findings from ETS’s documents framed baseline validity evidence to compare the findings from YBM’s documents and interview/questionnaire data that supported and challenged the TOEIC’s intended interpretations and uses. The overall findings represent the complexity of relationships within English constructs in international business workplaces and between stakeholders’ understanding and uses of TOEIC scores that were affected by contextual factors in South Korea, by highlighting the why and how. The findings can contribute to re-operationalizing the TOEIC’s constructs and call for including contextual factors in validation frameworks and studies. In particular, investigations of contextual factors cannot be overlooked to affect beneficial consequences in score use contexts. ETS and validation researchers may need to pay more attention to contextual factors and collect validity evidence in collaboration with relevant stakeholders.