Jack Maxwell and Joe Tomlinson: Model students: why Ofqual has a legal duty to disclose the details of its model for calculating GCSE and A level grades

On 18 March 2020, the UK Government cancelled GCSE and A level exams for students in England. The closure of schools and the need to slow the spread of COVID-19 made exams impracticable. But the Prime Minister confirmed that students would still get ‘the qualifications they need and deserve for their academic career.’ This created an obvious headache for public administration: the objective was to create a legitimate system of assessment, which could maintain confidence, without actual assessments.

Ofqual, the exams watchdog, has since developed a process for calculating grades this year, pursuant to a direction from the Secretary of State for Education. First, teachers will predict what grade a student would have achieved had they sat their exams in 2020 and rank each student relative to others with the same predicted grade. Second, exam boards will ‘standardise’ the predicted grades by applying a model developed by Ofqual. If the predicted grades in a particular subject at a particular school seem overly harsh or generous, the model will adjust those grades up or down, ostensibly to ensure that ‘grades are fair between schools and colleges.’

Ofqual has confirmed that the model will draw on a range of evidence, including historical outcomes for each exam centre, the prior attainment of this year’s students and of previous cohorts in each centre, and the expected national grade distribution for each subject. It has also confirmed that the model will consider prior attainment at a school, rather than an individual, level, so that students’ grades are not ‘predetermined by their prior attainment at KS2 or GCSE.’ But beyond this, as the Education Committee noted, ‘little detail has yet been published on Ofqual’s model.’

This is troubling for at least two reasons. First, there are credible concerns that the Ofqual model might systematically disadvantage particular types of students. High achievers in historically low-performing schools, or members of groups that systematically outperform their teachers’ expectations in exams (e.g. students from low-income families or ethnic minority students), might be given lower grades than they deserve. For schools with typically small cohorts and fluctuating grades from year to year, such as small special educational needs providers, historical data might not be a reliable predictor of future performance. This kind of disparate impact could ultimately call the legality of the model into question.

Second, without more detail on the Ofqual model, it is impossible for the public to know whether these concerns are valid, or to make informed representations to government about them. In its submission to the Education Committee, the Royal Statistical Society (RSS) noted that predicting and standardising grades, ‘fairly, at this scale, is extremely difficult and unprecedented.’ Without more information about Ofqual’s statistical methods, however, ‘neither the RSS, nor other external experts, can comment on their robustness.’

The essence of the anxiety about Ofqual’s approach is not new. Models – simplified representations of some aspect of the world which seek to generate analytical insight for complex decision-making – have played an important role in public administration for some time, particularly in areas such as environmental regulation and competition enforcement. And governments are often reluctant to disclose the models they use, due to concerns about confidentiality or abuse and circumvention. Indeed, there can be good reasons to keep a model confidential. However, this sits uncomfortably with the basic presumption that public power should be exercised transparently, which finds expression in administrative law doctrines such as procedural fairness, the duty to give reasons, the emerging principle of transparency, and the duty of candour. In recent years, the courts have proven willing to adapt and apply these doctrines to ensure that the use of models in decision-making is transparent, fair, and intelligible.

The leading case is R (Eisai Ltd) v National Institute for Health and Clinical Excellence. The National Institute for Health and Clinical Excellence (NICE) had issued guidance about the use of the claimant’s drugs in the NHS. A key factor in NICE’s decision was its modelling of the drugs’ cost-effectiveness. During its consultation, NICE had refused to give the claimant a fully executable version of the model. Without this, the claimant could not properly test the model’s reliability. The Court of Appeal held that this rendered NICE’s consultation unfair. Fairness required disclosure of the fully executable model, and NICE’s attempts to resist disclosure based on confidentiality and administrative cost were unpersuasive.

The courts have applied a similar approach in a string of cases (we provide an account of this line of authorities in a recent working paper.) The effect of these cases is that, where a model plays an important role in an administrative decision that might affect people’s rights or interests, the government comes under what amounts to a presumption of disclosure. It must disclose enough information about the model to enable people to regulate their conduct accordingly, to make meaningful representations about and understand the decision, and to meaningfully challenge the decision’s lawfulness in the courts. The courts are sensitive to competing values such as confidentiality, but the cases to date show it is generally difficult for the government to resist disclosure where the presumption otherwise requires it.

The Education Committee recommended that ‘Ofqual must be completely transparent about its standardisation model and publish the model immediately to allow time for scrutiny.’ Ofqual responded with an ambiguous commitment to publish ‘details about the model in the coming weeks.’ On July 27th, Ofqual wrote to the heads of centres committing to ‘publish the precise details of the model used on results days.’ It is still unclear what the actual extent of the information provided will be.

In our view, recent case law suggests that Ofqual is under a public law duty to disclose its model and to do so in a way that allows detailed scrutiny of the process. First, Ofqual has decided to consult the public on exam grades in 2020 and therefore it must do so fairly. Second, Ofqual is discharging a very important public function. Its decisions will determine the prospects of thousands of students across the nation, in both the short and long term. This requires ‘a very high degree of transparency in the process’ (Eisai at [34]). Third, the model plays a central role in the proposed grading process. It may directly determine many students’ ultimate grades, making the ‘robustness or reliability of the model … a key question’ (Eisai at [36]). Finally, the model must be disclosed, or at least significantly more information released, for anyone outside government to meaningfully scrutinise it, particularly in relation to the risks of discrimination discussed above.

It might be thought that disclosure would enable teachers to game the system on behalf of their students or work out grades early. It might also be thought that disclosure of and further consultation on the model would be too costly and time-consuming. The merits of any such argument will depend on the particular facts advanced on that issue and the situation at the time they are advanced, but the courts have indicated that they will be ‘very slow to allow administrative considerations of this kind to stand in the way’ of disclosure (Eisai at [65]). There is, of course, a need to keep students moving forward beyond their GCSEs and A levels but the decision-making process here will have far-reaching consequences for the lives of many young people. 

Jack Maxwell is Research Fellow in Public Law and Technology at the Public Law Project.

Dr Joe Tomlinson is Senior Lecturer in Public Law at the University of York and Research Director of the Public Law Project.

(Suggested citation: J. Maxwell and J. Tomlinson, ‘Model students: why Ofqual has a legal duty to disclose the details of its model for calculating GCSE and A level grades’, U.K. Const. L. Blog (28th July 2020) (available at https://ukconstitutionallaw.org/))