What is moderation for?
The purpose of moderation is to ensure that teachers are making consistent judgments about standards. In order to do this, they have to have a shared understanding about the expectations for each standard so that a particular level of achievement (for example, a Credit) is awarded to student responses with the same characteristics, regardless of who marks/grades them.
Moderation is an essential part of ensuring integrity in assessment tasks. It is through this process, particularly at the assessment design and point of assessment stages, that issues of assessment validity and reliability are identified and improved.
Validity is about making sure that the task assesses what you intend it to assess. That is, there has to be 'truth in assessment'. For example, if the purpose of a task is to assess students' content knowledge, but the task actually assesses synthesis of ideas, then it lacks validity.
Rubrics (criteria and standards descriptors) also have to be valid. For example, if a descriptor indicates that you are assessing a concept for a project, but, in reality you are also (without stating it) assessing use of literature, then validity is reduced. This means that students cannot confidently rely on the rubrics to guide their efforts. Validity is therefore about fairness and transparency in the design of tasks, criteria, and standards descriptors for students.
Reliability means that different assessors, acting independently using the same task description, come to the same judgment about a given piece of work or student response. Reliability therefore, is about fairness to students based on comparability between assessors. Rubrics associated with tasks also have to have reliability. This is tested when assessors use them to make judgments about grades. Even though complete objectivity between assessors is impossible to achieve, you should aim to make rubrics as reliable as you can - hence the crucial role of well-written and unambiguous descriptors. Assessors also need to be trained to use rubrics to judge student work, so that they come to the same understanding of the descriptors as other assessors.
Examples of some moderation processes include:
- involving all teaching staff in a unit in the development and review of criteria and standards descriptors
- cross marking/grading with follow-up meetings for discussion and comparison
- using one teacher to mark/grade all responses to a particular part of an assessment task, e.g. a section of an examination paper, the first two scenes of a play
- holding moderation meetings to confirm consistency of marking/grading across teachers; these meetings could involve:
- discussing any difficulties they encountered when making judgments, for example, interpreting a standards descriptor.
- developing solutions to these difficulties, such as altering the rubric to account for unforeseen and unintended student interpretation of wording.
- reviewing student responses and profiles of their results, in instances where there appears to be significant differences in marking/grading - this can assist teachers in fine-tuning their judgments so that they are in line with other teachers' judgments.
Moderation of results
A cornerstone of criterion referenced assessment is the practice of moderation. This practice is very important in ensuring that assessment is fair, transparent, valid and reliable. It is also essential in ensuring that the complexity of learning outcomes is increasing through a degree course.
There are three foci for moderation of assessment at the University of Tasmania, and each has processes which can be followed.
- assessment design (pre-assessment focus)
- making judgments (point of assessment focus)
- grading outputs (post-assessment focus)
The document, Processes for Moderating Results, was endorsed by the ULTC in 2010, and provides a table of these processes, as well as associated comments.
Casual Academic Staff
Training in making consistent judgments and subsequently moderating these
It is the responsibility of faculties to ensure that casual academics have had sufficient training in criterion-referenced assessment. This is important in building their confidence in making judgments about the standard of students' responses to tasks. They need opportunities to apply descriptions of standards to samples of student work from previous years. These samples should represent a range of standards. Ideally this would happen in a group situation so they can discuss their judgments with their colleagues. As well as training in making judgments using standards descriptors, casually employed academics need to be involved in a moderation process organised at course and unit level.
They may also be required to write rubrics - in this case, they will need training and practice writing standards descriptors.