GENERAL RULES:
o AVOID IT
o IF you require a quantitative response, for example, ‘…identify 5 types of maps’, try referring to quantity in the task description, rather than in the rubric (which ideally deals with quality).
o Make sure your rubric reflects the complexity of the ‘actions’ in the allocation of marks. Hopefully, the rubric will assess the quality of their response
o Assessing quantity in a rubric is akin to trying to assess quality (analysis, explanation, evaluation etc) in multiple choice questions. Difficult to say the least. We can only assume they have used critical analysis to arrive at the right answer but that is not necessarily true. Writing multiple choice and hinge questions is an art itself
SOME EXCEPTIONS
o Avoiding assessing ‘quantity’ is more important at higher levels of study, so if you want to use ‘quantity’ in Primary, Middle School or early high school, it is certainly a way of assuring the right ‘scope’ of response. However, there are some key provisos. (See below)
o If you are assessing a process, that requires the right steps in the right order, you are effectively assessing ‘quantity’. However, you will still be assessing the quality of those steps.
o Studying mathematics – can’t be avoided.
WHY AVOID QUANTITY IN RUBRICS?
Assessment is complex and one of the key parameters of effective assessment is to allow students to research in ways that allows them to dig deep to find the right answers, or if they become enthralled with a subject, follow their new-found intrigue to seek out answers that respond to their curiosity.
Any educator knows that you can get widely varied responses to tasks and sometimes, students will surprise you with the depth and quality of their responses.
Ultimately assessment is about the quality of the response. The student develops wisdom around how much is enough and at what point they have adequately supported a thesis statement. By defining the scope (e.g. use 3 sources) the student may not yet have achieved what they set out to do. When left open, the student response is still constrained by word count, however, they are also constrained by finding the right sources within the framework of the topic and word count.
Learning is also about judgement which is partly manifested in the scope of a student’s response.
WHY NOT?
The answers to this stem from the learning intentions.
Using the example above, first of all ask yourself why you want 5?
Is this a requirement of the curriculum? For the moment let’s assume it is not, just that students will understand there are several types of maps used for different purposes.
So, would 3 do if your intent is to simply make sure students know there is more than 1 type?
Or is one of the learning outcomes that students will know at least 5? Why – why not 6, or more? (Are there more – how many are there?)
So, to assess 5 maps, places an unreasonable emphasis on quantity.
Clear statements about what the student is to learn will clarify this. Re-visit learning outcomes or criteria to help define this.
What do we want them to learn? That there are 5 types of maps? Probably, no. More importantly, that there are different maps for different purposes
WHAT HAPPENS IF…
Again, using the above example, what happens if the task does ask for 5 types of maps? Nothing wrong with that (if quantity is key to the assessment) but is it reasonable to deem that quality is reduced if the student only does 3?
Do you assess the quality of the response or the fact that they covered 5 types (quantity)?
What happens if the student covered only 3 but their explanation of each type of map is excellent, resulting in an excellent understanding of types of maps and their purposes?
What happens if the student covered 5 but did so poorly? Is it equitable to disadvantage the student who covered only 3 (excellently) or to inflate the results of the student who covered 5 (poorly)?
It all depends on what you are trying to achieve in the assessment.
It’s all in the rubric!
If your rubric, rather than the task description, assesses 5, then marks must be allocated to this.
(You can, of course, ask for 5 in the task but only assess the quality of the response. It is likely you will still receive a spread of responses in which some students do less than 5. It is also likely that the quality of their explanations will be limited, rendering ‘quantity’ irrelevant.)
However, how do you determine the scope of responses across the levels?
1 is a ‘developing’
2-3 is a ‘consolidating’?
4-5 is an ‘extending’?
If the student can engage in the task at all, it is unlikely that they will provide only 1 example when asked to provide 5.
This would result in the greater majority of students achieving either ‘consolidating’ or ‘extending’. Generally, it is the quality of their explanation that will differ.
If the task asks the student, for example, to ‘identify and explain the purpose and characteristics of 5 types of maps’, what are the key qualities you are assessing?
o that they can ‘identify’ a map in the first instance (quality)
o that they can ‘explain the purpose and characteristics’ of each map (quality)
o (AND that they find 5 types (quantity))
Does quantity introduce an unnecessary scope? Further compounding this is the use of arbitrary divisions in each of the levels. It really depends on the importance you attach to identifying 5 examples. It could be assessed in the following way, for example:
1-3 is a ‘developing’
4 is a ‘consolidating’?
5 is an ‘extending’?
This would result in a greater emphasis being placed on quantity. So much so that a student who under most circumstances struggles with analysis and written communication receives an ‘extending’ for providing 5 types, however, only a ‘developing’ in the explanation of the maps.
Why would you do this? To provide incentive to the ‘developing’ students?
It is more likely that this would camouflage the true level of the student. At best, it would be wise to only use this assessment strategy in certain circumstances.
It would also be prudent to allocate a small percentage of marks to ‘quantity’ in this instance so that results do not indicate a ‘false’ standard that the student has achieved.