Exam Results and Negative Data

In 2012, Ben Goldacre made a really interesting TED talk called ‘What doctors don’t know about the drugs they prescribe’ and you can watch it here. I used to show it to my students because I thought they should know about it. Ben has gone off the radar a bit but I’m aware that when there was a big push behind the use of evidence in education, he was at the forefront of it. In fact he wrote this for the Department for Education after being asked by Michael Gove to explore the role of teachers in research.

On results day (and in the aftermath), I’m sure that if you were on Twitter, then you would have read about the excellent results Michaela School have achieved. Even if you were not on Twitter, you probably would have heard about them because Katharine Birbalsingh is a PR machine and many people wanted to draw attention to the positive results. And why not? They are great results and the school should be proud of their achievements. However, GCSE results (and A-Level) are pretty much a zero-sum game. That means as some schools go up, others must go down due to comparative judgement and the bell curve distribution of results.

How does this relate to Ben Goldacre and his fascinating talk? I know Twitter accounts and retweets aren’t academic journals and teachers aren’t researchers but if you were looking at the best way to get results after GCSEs, the only information you have are the positive results. And that makes sense…I mean, who would go on Twitter and say ‘our school saw a decline in our GCSE results today and those achieving 4-9, went down by 12%’?  But in order to make any sensible decisions on what you should do in school (or in the case of doctors prescribing medicine) you need the negative data as well as the positive data.

What if a school had tried to implement the Michaela Way but had found that in their school it was much more difficult to do due to the size of the year group? Or they couldn’t be as choosy on the intake because it wasn’t over-subscribed? Or the school buildings were designed in a way that they couldn’t have the communal lunchtimes that Michaela had? Or that they had gone full on Rosenshine but it just wasn’t working for some reason or another?

This isn’t making excuses for schools that have failed. What would be interesting is to go beyond the happy results in Twitter and find out what was going on in schools where they had a decline in performance. Or at least have an honest conversation about it so that there could be negative data included in the discussion. To be honest, they might have improved but the system only allows a certain amount of students to get a certain amount of grades (as discussed in this @teacherhead blog) but that in itself would prove useful. We would have evidence to suggest that the way grades are given out should change. It’s something I mention a lot on Twitter about how you have to keep moving forward in order to stand still and that seems rather unfair for our students.

I think about it from my own personal point of view. Over the past two years (and more) I’ve been inspired by the Learning Scientists and have introduced lots of retrieval learning into my teaching. In fact, I discuss meta-cognition with the students and go through the 6 effective learning strategies with them. I encourage them to elaborate, dual code, interleave, space their learning and so on and I stand at the front and give direct instruction.

So what happened to my results this year after two years of following the research? They went down. Not by much but they went down. Obviously there could be a number of reasons why this is the case but when you follow the evidence, you expect to see some improvement. Maybe I was just doing all of this anyway without me formally realising it (I probably was) or our students start from a very high base line (they do) and therefore the gains made are always going to be marginal but it does make me think about the number of schools (or teachers) out there who have made changes to their teaching and learning (following the research) and have actually seen no improvement.

It takes a brave teacher/researcher to reveal their negative data but if we want to move the profession onwards, we need to have all the information and I look forward to the time that someone puts a big banner on Twitter that hails disastrous results!!!

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create your website at WordPress.com
Get started
%d bloggers like this: