Rater Consistency in Rating L2 Learners’ Writing Task

Abstract

Abstract: Rater consistency plays a critical part in the rating procedure. Test scores will be unreliable if examiners are inconsistent in their rating and fail to agree with other raters on the relative merits of rating scale, severity and leniency and so on. Despite the difficulty in matching the standard, writing paper is widely used in various kinds of language tests because it can provide not only a high motivation for writing, but also an excellent backwash effect on teaching. For this reason, it is necessary to establish high consistency in the scores given by one rater (intra-rater reliability) and by different raters (inter-rater reliability). This article discusses rater consistency in essay evaluation conducted by some randomly chosen raters in the Faculty of English, theUniversity ofLanguages and International Studies, VNU and from that, some suggestions are made to improve the reliability in rating L2 learners’ essay writing.

Keywords: Rater consistency, intra-rater reliability, inter-rater reliability, holistic scoring, analytical scoring.