This study attempted to determine if English teachers' marking concurred with checklists they produced prior to the marking. That is, did the teachers assess what they thought they would assess? English is a relatively subjective field in which it is sometimes difficult for teachers to agree on items such as the outstanding features of a piece of writing. Because of this, it was hypothesised that there would be a significant lack of agreement between what individual teachers thought they would mark for and what they actually marked for, and between teachers. Seven very experienced teachers from one state senior secondary school and one private senior secondary school participated in the study. A typical Year 10 task and piece of writing was provided for teachers to mark. Before they saw the writing, the student's task was explained to the teachers and they all individually compiled a checklist indicating the features they would mark for when assessing the writing. The checklists and the marked writing were compared to find similarities and differences between teachers. The checklists and marking of each teacher were compared also in terms of similarities and differences. Although the sample size was small, it is of interest that the results of the study are supported by Diederich. In slightly less than half the cases, teachers did not agree on the order of the categories they would value. Neither did they value the categories in the same order in both their checklists and their marking. These results are significant for all English teachers because they indicate a need for further efforts to reduce marker variability.