To Grade or not to grade, that is the question...

Getting Garrulous about Grading 

A snapshot of one section of one of my grade sheets... 

If you're ever stuck for a conversation starter in a room of educators, you can be sure mentioning the 'g word' will get people going. Grading seems to be a particularly contentious issue that is perennial and generally (in the circles I move in anyway) criticised vigorously. 

I've been teaching for nearly 25 years at the time of this post, and during that time, my feelings on the subject have vacillated wildly, but—and maybe this just the wisdom of age—while appreciating the various sides of the arguments, I can't help but return to the fact that to me, instinctively, intuitively, it just makes sense. But as in most issues of this nature, it's how you do it that counts. 

Many teachers who tell me they don't grade are just playing semantics, if they are any good (and most are) they are making judgements about the efficacy of the work their students do all the time. I’ve seen all sorts of attempts to avoid it, but other than prose comments (which takes ages, and students generally ignore) they’re all still essentially (and will be) forms of scoring, whether you use a rubric, continuum, thumbs up/down, traffic lighting, or even 'impression marking' unless you’re going to rely on screeds of text you'll end up with 'grading' in all but name. Unless you just do ... nothing? Does anyone think that is better? 

All grading is, is quantifying the qualitative, it's giving an assessment judgement a number/letter in order to make using it as a rich source of data easier. That's it. 

Hypocrisy

My main protestation about the use of grading for years has been its inauthenticity, its artificiality. After all (like examinations—which I still maintain intense disdain of) how many of us get 'graded' in the real world? For most of us the last day of exams, is the last day we'll be graded on anything. Once you stop being a student and enter the world of paid employment, the grading ceases. 

Unless of course you reenter the world of education as a student, in which case it can feel like a rude reawakening—at least that's how it felt for me. When I did my Master's degree a few years ago, I really resented the grading component (despite getting very good grades) for me it changed the whole dynamic, it undermined the focus. But that is because of the way it was graded... it felt like too much too late, and at least once, fundamentally unfair. But since returning to teaching (having been in a primarily coaching role for 8 years), I’ve returned to it with gusto. 

So to its irrelevance; actually if you think about it, in many ways grading is effectively a proxy for $, for adults $ is commonly an indicator of success. In many careers appraisal is common, and the results can often have $ implications. Even if you're not in a career where your 'performance' is assessed and rewarded financially, there are many where your 'performance' is your work to date, your reputation, and your performance at interview; the reward is the 'assessment' of your success, perhaps employment, and/or promotion. Students whose work is graded get a taste of reality, accountability, a 'real world' experience; in the world outside school, your work will be judged, and depending on how successful it is deemed to be, you will be rewarded. And if you're not (you don't get that job/promotion) you use that as feedback, you reflect, you make changes, and you move on. Life is a series of assessment experiences, the results of which shape the people we become. They may not always be represented (directly) with a number, but they are frequent, they are generally (relatively) low stakes, and preparing students for a world where this is the case is essential. 

There are some authoritative voices out there who defend the use of grading, and—of course—a considerable amount of research: 

Students Should Be Tested More, Not Less (Atlantic) 

"Complaints that excessive testing detracts from learning tend to be aimed at summative testing. As summative tests do not teach, and classroom hours spent engaged in summative assessments detract from hours a teacher has to educate her students, those complaints are probably well-founded.

“Formative assessments,” on the other hand, are designed to discover what students do and do not know in order to shape teaching during and after the test. Formative assessments are not meant to simply measure knowledge, but to expose gaps in knowledge at the time of the assessment so teachers may adjust future instruction accordingly. At the same time, students are alerted to these gaps, which allows them to shape their own efforts to learn the information they missed.

"Formative testing at its best is low-stakes and high-frequency."

"When teachers expose students to frequent low-stakes tests in order to reveal gaps and foster active, continuous engagement in the material, students are given more ownership and power over their education."

Making the Grade: What Benefits Students? (Educational Leadership)

"Grading enables teachers to communicate the achievements of students to parents and others, provide incentives to learn, and provide information that students can use for self-evaluation."

There is a great deal more, but the problem is, very few of the methodologies used in these studies actually (by their nature) use FLS, they assume a model where either high stakes examinations are the focus, or one-off summative testing. That is not what I'm talking about, the reasons why could be the subject of many ranty posts, but I'll leave that to others more qualified and more authoritative than me, no I want to focus on frequent, low-stakes, grading. 

Studies on the Relationship Between Frequent Low-Stakes Testing and Class Performance

A Meta-analytic synthesis of data from 52 independent samples from real classes (N = 7864) suggests a moderate association of d = .42 between the use of quizzes and academic performance. Effects are even stronger in psychology classes (d=.47) and when quiz performance contributed to class grades (d = .51). We also find that performance on quizzes is strongly correlated with academic performance (k = 19, N = 3814, r = .57) such that quiz performance is relatively strongly predictive of later exam performance. We also found that the use of quizzes is associated with a large increase in the odds of passing a class (Abstract) [paywall]

Sotola, L.K., Crede, M. Regarding Class Quizzes: a Meta-analytic Synthesis of Studies on the Relationship Between Frequent Low-Stakes Testing and Class Performance. Educ Psychol Rev (2020). https://doi.org/10.1007/s10648-020-09563-9

FLS (Frequent Low-Stakes) Grading

I haven't alway known that what I do has an acronym, but I've been instinctively using this approach for as long as I've been teaching. It just makes sense to me, low stakes, frequent grades that students know they can improve if they make the effort are very useful. Much more efficient than a prose laden alternative that dominates in schools and systems that purport to be 'grade-less'. 

FLS Transformed by Tech

I guess one thing that really excites me about this approach is the way tech can transform it. In the 'old days' the data that builds over time would only really be available to the teacher, I would share it with students and their parents, but it was tricky, as I needed to preserve the privacy of the rest of the students in the class. This invariably meant either making/printing individual copies, or fiddling about with folding over sections of sheets to hide information, even then a hard copy is quickly out of date if you're using FLS. But the advent of online platforms has radically changed this. For example with the platform we use in Middle and High School (Teamie) students have access to their Markbook, so they can see the data, and act on it accordingly. This runs the gamut from students who use it to moderate their efforts so they can work towards a pass (and maximise their time for other pursuits/subjects) or those who use it to constantly refine and revise specific tasks until they are as good as they can make them. It puts the student in the driving seat, and gives me rich data to use as a  basis for my teaching, and my conversation with each of my students about their next steps. 

Fortunately for me, it wasn't hard to find some clever people who have summarised this approach and its benefits for me: 

Benefits of low-stakes assignments

• Gives students a realistic idea of their performance early in the term, enabling them to seek appropriate resources as needed
• Opens up lines of communication between students and their instructors, and may increase students' willingness to ask for help
• Allows instructors to direct students to resources if they need further assistance or support
• Gives students an opportunity to be active participants in the evaluation of their own learning
• Increases the likelihood that students will attend class and be active and engaged

"These exercises are low stakes, they can improve learning outcomes without increasing student anxiety. "

"Frequent, low-stake assessments as opposed to infrequent, high-stakes assessments actually decrease student anxiety overall because no single test is a make it or break it event."

"Feedback should be given often so that students can benefit from having multiple opportunities for improvement. Though given less weight, low-stakes assignments may be similar in type and kind to high-stakes assignments: they tend to reflect the kind of work students are going to be expected to do for a final exam, paper, or other summative project. All in all, early feedback is one of the most important contributions faculty can make towards helping students succeed in their classes and make critical progress…"

(Sarah Jones, Michigan State University)

Record & Flip - Blended Screencasts

Record & Flip

Record it, then flip it, simple!

Screen recording, or Screencasts are one of most most effective teaching tools in the arsenal of a teacher who is fortunate to be work in a 'technology enhanced learning environment' but, if the devices you have to hand are laptops, not tablets, than expecting your students to create a screencast can be more of a hassle than it's worth. 

That is unless you know how to record and flip. 

I've used demonstrated this technique before in a context of asking students to model a skill with apparatus, such as how they can measure an angle with a protractor, but with a little imagination it's not difficult to see how this could be used in other ways:

• Mapping skills in Humanities
• Rationale for a design proposal in Design & Technology
• Description of the significance of imagery in the Visual Arts
• Mind Mapping relationships and connections
• Reflection on ideas and opportunities for development in a flash draft in English
• Reflection/critique of a passage/excerpt in a printed book/magazine
• Annotation of musical notation to indicate understanding of the structure
• Annotation of diagrams, graphs, and charts
• Explaining a strategy or process in solving a Mathematics problem
• and many more...

All the students need is a whiteboard or a sheet of paper, and to position it as shown above. They can tap the spacebar to start and stop the recording. 

Once the recording is finished the student can flip the video horizontally and vertically, then review and trim* the video is necessary before sharing it with the teacher. 

An Example from Mathematics

Here's one I did earlier...**

*If the student has done a great deal of  'umming and ahhing' they can delete the segments of the video that are unimportant, but most of the time this is probably unnecessary as you're not looking for a highly polished artefact here, and hesitation (when and why) may well be useful information in and of itself.

Student Example

** Disclaimer, the hesitation you see in this video was intentional in order to create a sense of authenticity, honest, it's true! 

Deliver us from tedious tests and rubrics

via hippoquotes

Assessment drives everything educational. So, not surprisingly, assessment is the biggest factor in terms of planning the use of tech in effective ways. This means that it's critical to ensure that we use a varied range of assessment strategies, which is where I find a surprising lack of options.

Why do so many teachers assume that only rubrics and tests are suitable for assessment? Sure they have their place but only within a suite of assessment strategies...

It feels to me like every educational reference I read or hear about, especially in tech circles, assumes that the only viable option has to be a rubric. I don't mean to denigrate any particular assessment tool—clearly rubrics and tests can be effective assessment tools, but when they dominate, they have an unfortunate tendency to diminish the importance and efficacy of all of the other tools that are available. It is depressingly common to me that in virtually any educational context (classroom, conference, online) when the conversation inevitably turns to assessment, the question seems to default to, 'what rubric or test will we use?' rather than any awareness that there are a plethora of other tools and strategies that could be just as effective if not more so.

Now I am concious that I may be overstating my point, after all, I have to confess I don't hate them, I hate the way they are so often assumed to be the only option worth considering. I loathe the majority I see that are poorly conceived and poorly written. They are often bloated verbose attempts at teasing out questionable differences in attainment, many that seem to be based on the assumptions that just adjusting superlatives is sufficient, like well, very well, independently, with assistance...

Of course I'm not the only one who has a problem with rubrics:

The most famous of whom is probably Alfie Kohn who speaks to the false sense of objectivity and how rubrics have misled many.

And I really like Joe Bower's take on Rubrics, in 'The Folly of Rubrics and Grades'

"Grades and rubrics are a solution in search of a problem that will further destroy learning for its own sake.  

It’s been five years since I used a rubric. I simply don’t need them, nor do my students.
Rather than spending time asking how can we grade better, we really need to be asking why are we grading. And then we need to stop talking about grading altogether and focus our real efforts on real learning."

Most of the rubrics I've seen could be easily replaced by a continuum, at least then all you would need to is define the extremes, but and I guess this is a statement about teaching as a profession, far too many teachers use the term 'rubric' as if it is synonymous with 'assessment tool'.

Rubrics are one of many ways to assess learning, and they are used far too often. Used well a rubric can be a powerful assessment tool, but in my experience I rarely see them used well, and I often see them used inappropriately.  So, yes, they have their place but only within a suite of assessment strategies...

Here's one way to use a rubric well, by making it more student centred' this way the teacher defines a central standard (eg a level 3 on a 5 point scale) and then leaves the students to define and justify the level they feel there work sits in comparison to that, (above or below, or in the middle) with examples.

There are other ways to assess... 

Next time you're assessing, at least consider some alternatives to rubrics. Now before someone accuses this of being more new fangled thinking, here's some out of the Ark:

But one of my favourite summaries of assessment strategies and tools, is this grid from the PYP:

Unfortunately the PYP is allergic to the term 'tests' and (somewhat simplistically in my opinion) assume that all tests can be summarised as 'checklists'. Still, if more educators made more effort to tick all the elements in the above diagram in one year everyone would be a winner. I've always found this matrix from the PYP to be particularly useful to illustrate this, although you may be surprised by the omission of tests from this grid, I believe they (somewhat disparagingly?) categorise these as 'check lists':

Do less, but do it better.

Now of course it's highly possible that teachers are unaware of the wider range of assessment tools they use effectively almost everyday, such as the ad hoc/informal conversations (conferences in the jargon) with students every day, to spirited class debates (not lectures) that utlise skilful Socratic strategies, which are in and of themselves valid assessment tools. The problem is that I think these are seen as somehow inferior to a "proper" test/rubric. All this does is create a lose/lose scenario for the teacher and the student. Rather than focusing on tests and rubrics, wouldn't it be better for everyone if we were to embrace a much wider tool kit when it comes to assessment? To see them all as valid/powerful, maybe that conversation/conference was so effective that adding a rubric or a test is not only unnecessary but possibly even counter productive?

I think if you had asked most teachers why it is that they rely so strongly upon rubrics and tests as opposed to all of the other powerful forms of assessment, I think you would find that they would point to one sad fact; they feel they need paper with marks on, that they can attach a grade to, so they can point to it as being hard evidence of their assessment judgement. While there is clearly a place for this kind of formal (usually summative) judgement, in my experience it is far too frequent and far too common. Teachers could do themselves a favour and do their students a favour by focusing on the goal of learning rather than the need to have a hard artefact to present evidence of every stage of progress.

What if instead we were to focus on the goal, that is, as long as the assessment tools you use allow you to provide effective individual feedback to the student and enables them to progress in their learning point where they are improving compare to their previous level of competence (ipsative assessment), then the goal has been achieved! So why not work a little smarter and use a range of assessment tools that are a far more varied. In so doing you create a classroom environment that is more dynamic, and far more effective for both the teacher and the student.

So what does this have to do with edtech?

From my perspective, a classroom that exploits a wide range of assessment tools is a much richer environment within which to be able to integrate digital tools that can truly enhance and transform the way teachers teach and the way the students learn, and demonstrate the extent to which they have mastered the skills, knowledge and understanding that is truly the point, not just in ways that can be measured quantitatively on another test or a rubric. You don't have to look much further than an early childhood classroom to see this in action. Why? One thing these very young students can't do is demonstrate their understanding via tests or rubrics, which opens up a whole range of extremely rich engaging ways of demonstrating skills knowledge and understanding that would benefit many students that are considerably older, 

Transforming Assessment for/and/of Learning

Stop Marking and Start Giving Feedback

Here's a magnificent example of how assessment can be transformed by the effective use of digital technologies.

Replace, Amplify or Transform - RAT

It is so easy for even powerful applications of ICT to be little more than replacement for traditional tools and methods, maybe amplified—by that I mean, maybe there are benefits in terms of amplification, like speed, or efficacy, maybe the activity is more motivating, maybe it uses less paper, but is it really changing learning; or the kinds of teaching you can do?

If not, then it's not transformative, and it's not what I am aiming to see in the classrooms of the teachers I 'coach'.

So, one of our Grade 5 teachers is transforming formative assessment, and he's doing it by cleverly combining two powerful tools, in a synergetic way that amplifies the potential and possibilities of both, here using Google Doc comments and QuickTime screen recording.

Now he's been using screen recording for some time to give feedback to his students, and then he started experimenting with getting his students to use the same method to create a 'learning talk' to explain the rational behind their writing choices for him, great, but the problem with that is you now have 22 x 2-3 minute videos to 'mark' and the problem with video compared to text, is you can really 'skim' it, and annotating it? Not easy, certainly not as easy as paper.

In short video is great for pushing content to kids, but it sucks for pulling content from kids.

Unless… you get the kids to use each other as resources, what I call P2P (peer to peer), what Dylan William calls 'Students as resources for one another'.

So, have a look at this video, by an 'ordinary' student, on an 'ordinary' day in an 'ordinary' classroom at UWCSEA, but one who is transforming learning of her peers.

SAMMS

It's worth pointing out exactly why this is transformative, because it leverages 4 out of the 5 unique affordances of ICTs, that I call 'SAMMS'; this work is:

Situated - This student can start the work at school, continue at home, and seamlessly return to at school no problem, any space, any place, any time, that suits her.

Multimodal - Text and speech = win win.

Mutable - the workload implications of acting on feedback are minimised compared to paper. Feedback annotation on paper would effectively require the student to start again and rewrite the entire thing from scratch - the my ability of the screen mean edits and revisions can be made quickly and simply, and supported by supportive proof-reading tools.

Social - This document has effectively become a micro-community, two students and one teacher, but it's not one way. Following the feedback, any of the 3 parties involved can pitch in with comments to clarify, redirect, reinforce, resolve… Invite others to join in? the possibilities are endless!

Ownership of Learning - What does that actually look like?

"Students as owners of their own learning"

Leading teachers through the 'key strategies of formative assessment' (Wiliam, 2011) the one I wrestle with the most is the last one:

1. Clarifying learning intentions
2. Eliciting evidence
3. Feedback that moves learning forward
4. Students as learning resources for one another (peer assessment)
5. Students as owners of their own learning (self assessment)

Only because, in a 'traditional' learning environment, such as that I commonly observe in Cambodia, one things that really stands out is the fact that it is commonplace for every single child in the class to work on 

exactly. the. same. thing.

In desperation I searched for a basis to question the validity of this, and the ownership element struck me as an obvious candidate.. except that no matter how much I read I just couldn't find a way to get the literature to say what I wanted it to say, that ownership requires autonomy.

The nearest I could get was that ownership involves, "metacognition, motivation, interest, attribution, self-assessment" (Wiliam & Thompson, 2007)


So ownership is interpreted as meaning self assessment, but what is less clear is the extent to which this is even possible unless the students are able to produce outcomes which are 'unique'.

It is my contention that a situation where every student is engaged in the creation of an outcome which is completely identical to the one next to them, is not one where 'ownership' can really be said to be present. A common practice in Cambodia is to give all students identical tasks to complete, like recreating a document on screen based on an identical hard copy, the goal being for all students to produce something is close to that as possible.

Based on this kind of model, students can self assess, the success criteria are clear, as is the learning intention etc, but my contention is that there can't really be 'ownership' if the outcome is not unique to them in some way.

So does 'ownership' relate to unique outcomes, or not ... in desperation I contacted Mr Wiliam himself, and he kindly responded with,

"... all learning outcomes are personal, since the only way one can learn anything is to make it one's own. 

However, it seems that you are asking something slightly different, which is can the learning of imposed goals ever be truly "owned". And I think the answer to this question is yes. The account of the work of Deci and Ryan in the attached chapter (p. 1081) provides some insight here. As they say, extrinsically motivated learning can still be autonomous..."

So off I went and read the kindly attached article thoroughly, not just the bit he referenced but the whole thing:

Wiliam, D. (2007). Keeping learning on track. Second handbook of research on mathematics teaching and learning, 1053-1098.

And what I found was intriguing, you see the very study Dylan cites in defence of the importance of self assessment, actually places as much (if not more) emphasis on autonomy...  Reading further back in the article, to the beginning of section about which this conversation focuses. Reading the study described on p 1078 I can't see how the results could be attributed to self-assessment alone, the clear emphasis on increasing student autonomy has to have been as influential, if not more so. It's strange that the effect is automatically assumed to be solely due to self assessment, without any attempt to differentiate between the influence of those two distinct elements.

"The scores of the students taught by the teachers developing self-assessment improved by 15 points—almost twice as big an improvement." p 1081

Surely they were developing student autonomy to at least as great an extent as their abilities to self assess? In fact it seem to me that the two are synergetic; autonomy facilitates self assessment:

"Finally, in the last 10 weeks, students were allowed to set their own learning objectives, to construct relevant mathematical problems, to select appropriate manipulatives, and to identify suitable self-assessments."

And I'm pleased to report that things are already changing, check out the CHANGE, last week:

So, my respectful apologies Mr Wiliam, but I just can't encourage our teachers to keep managing classes of automatons mindlessly churning out identical outcomes, like a production line.

I think it's a particularly important distinction in the context of Technology Enhanced Learning (TEL), as in contexts where teaching is dominated by traditional technologies like the photocopier (class worksheets), there is clearly a lack of autonomy (even if there is motivation, for whatever reason) ... Set them free!

FOCUS Lessons

Having just returned from a week of teacher training in Phnom Penh, at the most magnificent CCF I have to confess that within 15 mins of my first lesson observation (of a planned 6 lessons in 2 days) I was seriously struggling with the point of it.

Why observe without acting?

Well the truth is I just couldn't just sit there and watch, knowing that this lesson, NOW could be better. If it can then, let's ACT, lets do what we can, while we can, NOW.

And a FOCUS lesson was born—well the acronym emerged during a wonderful massage later on, but the practice was already a fact; only it was a practice that I was struggling to describe to my colleagues, when I confessed I'd abandoned the observations, and lab-sites that they so dutifully (and professionally I might add) pursued.

So, what did I do instead?

Well, I attempted to describe it as an observation, cum intervention/co-teach/sharing/skilling/teaching/reflecting/advising/adjusting lesson.

To which they said something like, "Well you're going to have to come up with a better way of describing it than that."

So I did, and I have. And in case you're wondering, it was a GREAT massage.

So I call them FOCUS lessons, largely inspired by Dylan Wiliam's (2011) 5 key strategies for formative assessment, which I pretty much use for almost everything... Just substituting 'learners' for 'teachers' Yes, I know teachers that are any good have to be learners, but you know what I mean...

1. Clarifying, sharing, and understanding [teaching] intentions and criteria for success
2. Engineering effective classroom discussions, activities, and [teaching] tasks that elicit evidence of [teaching] 
3. Providing feedback that moves [teaching] forward 
4. Activating [teachers] as instructional resources for one another 
5. Activating [teachers] as owners of their own learning

(Adapted from Wiliam (2011). Embedded formative assessment)

And being a huge fan of acronyms I ended up with:

F: Feedback & Feed Forward
O: Observe (Learning not just teaching)
C: Co-teach & Constructively Criticise
U: Upskill & Unlearn
S: Suggest & Share (good practice)

Meta ACRONYMS

Alphabetically Coded Reminder of Names You Misremember
A Contrived Reduction Of Nomenclature Yielding Mnemonics
A Concise Reduction Obliquely Naming Your Meaning
A Clever Re-Organisation to Nudge Your Memory