The Online C-Test: Evaluating a discipline-specific academic literacy competency assessment tool

Abstract

This paper examines the use of an experimental online post-entry language assessment (PELA) tool during the first year of an undergraduate Information and Communication Technologies (ICT) course provided by an Australian university. The research was conducted in response to English language teaching staff, subject lecturers, and the Australian Universities Quality Agency (AUQA) expressing concern that students were engaging with the ICT course without appropriate English language proficiency. The response to these concerns was for the authors to implement an assessment of English language competency to establish the extent of the concerns, and to provide recommendations as how these issues could be addressed. Three forms of testing were employed, a reading/summarising exercise, a listening test, and the online discipline-specific literacy assessment tool. More specifically, this paper details the design and evaluation of the latter online assessment tool utilising the ‘C-Test’ procedure. The outcomes of this research were twofold. First, it identified that as many as 39% of students were likely operating with a lower level of linguistic proficiency than was considered appropriate for successful engagement with the course, which then allowed the authors to provide a list of recommendations to help alleviate this issue. Second, while extensive research suggests the C-Test is a reliable tool to gauge linguistic competency, our research found only a weak correlation between students’ C-Test results and their results on a concurrent reading/writing (summarising) test. This result points to a need for more rigorous research into the concurrent validity of the C-Test and the way that it is implemented online as a measure of discipline-specific linguistic competency.