The system and the student

This blog post is written with the benefit of hindsight and the luxury of not being accountable. I don’t pretend to have warned this would happen or have any easy answers.

The problem with grade awarding this year was that we tried to act as if exams were being sat when they weren’t. A system was then set-up to try and replicate the overall pattern and feel of an examination year, with only a fairly loose relationship to the performance and potential of individual pupils.

Actually, this isn’t exactly unique to 2020. Due to the desire to keep the overall results looking ‘right’, there is usually, year on year, a large volume of error in individual results. The key difference is that people are inclined to believe the exam result over teachers’ or schools’ predictions. This year, there were no exam results, and the algorithms used were applied to very imperfect proxies. But there is always an algorithm.

Long before this year, we as a country chose to prioritise getting a set of results that look believable in terms of headline numbers over accuracy (and fairness) regarding individual pupils. This is a classic case of prioritising the system over the people within it. The choices made during the crisis of Covid-19 have accentuated an existing problem to extreme proportions.

Now is a good time to reflect whether this is the right design principle.

There is value in the integrity of the system, it gives currency to the individual results, but only up to a certain point. There is an argument now in favour of generosity to the individual young people inside the system. People have worried, understandably, about how this year’s results will be compared to last year’s and next year’s, as different cohorts compete in the job market. That is now a moot point. No one will ever believe that this cohort’s results are comparable to any other year. That ship has sailed.

Our priority should therefore be ensuring that each young person gets to the destination they had waiting for them. This cohort have lost so much with Covid-19 - let’s not take their plans and aspirations away from them too. What does generosity cost us at this point?

This could be achieved in a number of ways: we could refer to centre-assessed judgements. These are more generous, because teachers see potential over time; exams see only performance in the moment. But schools’ assessments are not random or baseless. If some young people end up with higher results than if they’d sat an exam, let’s fill the gaps in their knowledge retrospectively, not penalise their futures.

We could also honour offers for courses young people received before their results. To achieve that, we may need to help universities and other destinations to support more places. So what if they have more students in the year ahead, would that really be bad if the institutions have the resources?

These measures need to be applied retrospectively to A levels and prospectively to GCSEs. If that work cannot be done in the time remaining for GCSEs, there is an argument for delaying their announcement.

There is urgent work to be done as young futures hang in the balance. But there are longer term lessons to be learned.

When you use any new and complex model, you never really know what the results will be. It seems that, by relying on centre assessment for smaller groups and statistical adjustments for larger groups, the model is more generous to people in smaller groups. Private schools students are more likely to be in smaller groups.

It is reasonable to accept that this was hard to predict, especially when working rapidly in a crisis. This is a strong argument for ensuring all grading models are open, transparent and published long in advance. There are many capable people in schools and colleges across the land, and in organisations around them, who can test any algorithm and work together with those in power to discover and correct any issues.

The current system has operated for many years with an unspoken principle. That is, that individual inaccuracy is a price worth paying for confidence in, and the stability of, the system. It is a kind of consensual fiction that everyone in education is required to participate in. Individual young people don’t always get the results they deserve but things feel stable and rigorous.

Right now we risk piling disadvantage on the already disadvantaged. The more privileged can cope with errors: they can appeal, resit, lobby or use their networks to explain the results and get where they need to go. Disadvantaged pupils need the top marks that they’ve worked so hard for. Their grades are their objective passport to future success, they enable them to cut through when they will at points in their life be subjectively judged by their background, class, postcode.

We need to design a system of assessment that ensures those who most need it get the most accurate results. This should be principle number one. And of course, the results are only the product of all the work that has gone before: teaching and study stretching back years. If you want fair results, you need more than equal inputs. You have to weight resources to counterbalance the unfairness elsewhere.

The use of prior attainment as one of the driving forces in the exam system is particularly problematic. It acts to erase the possibility of confounding expectations. And who does that benefit? Those for whom we already have high expectations. How does a student, or school get to prove us wrong and leave their history behind?

Can we design a system that puts fair results for those who most need them ahead of other considerations? I am not an assessment expert. It is a complex specialism, but it can’t be beyond our grasp. I believe in exams as a potential tool for social justice as they do not depend on judgement. But that is not their current effect.

When proposing remedies, we should be wary of solutions which work for a pandemic year but will be less than ideal in normal years. Yes, coursework or modular exams would be more resilient to cancellation. But they also favour the privileged in their own way. Remember, the problem this year was not that we had exams. It is that we didn’t have exams. All that was left was the system we used to adjust and shape the results of the exams - and thus its effects were impossible to hide.

Surely the time has come to challenge the principle and build a fair education for all.