Analysis of the English Test Level A (PRETCO) for Vocational Students in Yunnan, China PR

Abstract

This study analyzes the content of the English Test Level A (PRETCO), a standardized vocational English assessment in China. The analysis applies Messick’s (1996) Validity Framework and Bachman’s (1996) Communicative Language Ability Model to examine PRETCO test papers from 2021 to 2023. A mixed-method approach is employed, combining thematic analysis, grammatical analysis, and content validity assessment. Item-Objective Congruence (IOC) is used to measure the degree of alignment between test items and vocational English competencies. The findings address both research objectives. First, the content analysis reveals that the Listening and Reading sections align well with vocational English tasks, while the Translation and Writing sections show inconsistencies in task design and scoring reliability. Grammatical analysis indicates a strong focus on basic structures such as noun forms and verb tenses, with limited integration of complex grammatical constructs. Second, the content validity assessment shows that PRETCO-A achieves a moderate-to-high level of construct alignment (IOC = 0.78), indicating that the test reasonably reflects communicative competence required in vocational settings, though further improvement is recommended (Messick, 1996). These results support the enhancement of PRETCO-A and provide guidance for educators and policymakers to align vocational English instruction with assessment standards.

The findings indicate that while the Listening and Reading sections align well with vocational English competencies, the Translation and Writing sections exhibit inconsistencies in scoring reliability. Content validity analysis shows that PRETCO-A demonstrates a moderate- to-high level of construct alignment (IOC = 0.78), indicating reasonable but improvable validity in measuring communicative competence needs in vocational contexts (Messick, 1996) These findings contribute to the ongoing development of vocational English assessments, ensuring stronger alignment with professional communication demands.