Empirical research into judicial review has recently started to draw attention, as Brian Christopher Jones points out in his careful and thoughtful critique of its use and utility.

Jones is sceptical of some of the claims made by this empirical research – which, at least as of late, has tended to take aim at claims of expansive judicial over-reach, either in general or in relation to certain provisions such as those in the Human Rights Act. Much empirical work, for Jones, seeks to underplay the evidence for over-reaching judicial power, and presents an account of judicial review which is relatively unobtrusive and pedestrian. In doing so, empirical work takes too narrow an approach, focusing as it does on outcomes and raw statistics, and treating all cases alike. This in turn ignores importance of the reasoning deployed, the particular importance of landmark cases, the soundness of the constitutional principles behind those cases, and so on. Incidentally, the dimensions considered relevant in empirical work, we are told, favour the idea that there is “nothing to see here”; those side-lined would favour the argument that judges are too activist. The result is that the empirical literature is, perhaps intentionally, unbalanced and lop-sided and should be treated with greater scepticism than it is at present. 

Is this a fair criticism of empirical research on judicial review? I suggest that whilst there are important lessons to bear in mind from Jones’ piece, the crucial utility of empirical work goes unacknowledged. This post will try to explain how. 

It is important to acknowledge some concessions at the outset, many of which are rightly brought up in Jones’ piece. Firstly, empirical research does often pay little attention to the relative importance of cases. As Jones notes, a case such as Evans v Attorney General is more important – for most purposes – than an everyday planning permission case decided by the Administrative Court. Sometimes the decision to give all cases, landmark or otherwise, equal weight in an empirical study is a conscious decision by researchers, to ensure as value-free a dataset as possible. But even in these cases, the importance of ‘big’ cases should be borne in mind, and factored in to any wider conclusions proffered. 

In addition, researchers must always remember the limitations inherent in empirical research: firstly, it will almost always be restricted in practice – by the time and resources available to the researcher, by potential methodological difficulties arising, by problems with coding and analysis, and the like – which means empirical work can only ever offer a partial insight into a legal issue, albeit hopefully one which is robust enough to be reliable. Further, empirical research tends to be a fairly blunt tool. Whilst empirical work can shed light on, or offer insights into, concepts such as judicial decision-making and the realities of political litigation, even the most sophisticated research cannot hope to fully explain or completely understand such complicated and multi-faceted phenomena. 

So where does the utility of empirical research lie? I suggest that the answer depends on what each piece of empirical work is aiming to achieve, and what kinds of claims it is responding to. 

Jones is right: typically, empirical analysis generally tells us little about cases such as Evans or Jackson, which are important because of what they say about thorny constitutional questions. They are controversial and contestable and it is no surprise to find that such cases fill pages of the relevant textbooks. Law students care about Evans and Jackson because of their great constitutional significance. And so they should.

Empirical researchers do not deny the constitutional importance of these cases – but they are sometimes not interested in the constitutional importance of these cases. (On the other hand, some kinds of empirical research, particularly that which employs qualitative rather than purely data-driven methods, is very much interested in answering these kinds of questions!) Whilst of course dependent on the nature of the study in question, empirical researchers – at least those cited by Jones – are generally more concerned with asking how the legal system works in practice. From such a perspective cases like Evans and Jackson do not necessarily matter more than any other. The most useful empirical studies focus on answering questions which can usefully benefit from an empirical angle. As Jones notes, all legal questions do not fall into this category.

But many do, including those raised and relied upon by governments in justifying legal reform. Often, empirical research is responding directly to another empirical claim, and one which is being used to justify legal or constitutional change. Empirical researchers are playing the same game as those they are responding to. Some examples may help to illustrate this.

When the Independent Review of Administrative Law made claims that Cart judicial reviews were eating up disproportionate time and resources, this was relied upon by the government to justify abolition of the Cart jurisdiction. Various pieces of empirical work sought to evaluate IRAL’s claim (it did not help that the IRAL report was muddled with some fairly fundamental statistical errors). Of course, this kind of research says nothing about whether the outcome of Cart was correct, what its implications are for the separation of powers, or whether it is evidence of burgeoning judicial activism. If these kinds of arguments were said to justify reform, empirical research would indeed be ill-suited to argue against it. But the claims which were made, in practice, were exactly the kinds of arguments to which empirical research could provide an answer.

When the Judicial Power Project made claims that the Human Rights Act was being frequently used to strike down statutory instruments, empirical research showed that this was not the case. Again, this kind of research did not, and could not, make any direct claims about the constitutional soundness of the HRA or whether it represents an appropriate balance of power between judges and politicians. It could, on the other hand, question whether proposals put forward by the JPP (unsurprisingly enough, to roll back the HRA) were justified in light of the lack of evidence behind their original statement. The strength of the original claim was something which empirical work could usefully help to determine.

Sometimes, empirical work does not definitively answer a particular question, but can contribute to properly framing the debate. For example, arguments for reform of section 3 HRA often rested on the idea that the provision allows judges to depart radically from ordinary canons of interpretation. The government relied on eye-catching dicta and some fairly extreme cases to justify its conclusion that section 3 goes too far and ought to be reformed. By contrast, empirical research looked at how often, and under what circumstances, section 3 was used in practice over a number of years. This study showed that in a large number of cases, the use of section 3 was virtually indistinguishable from the exercise of ordinary statutory interpretation. 

This kind of finding – that section 3 was not being used to radically alter interpretation across the board – did not immediately close down the debate on section 3. It did not, on its own, show that reform was not a reasonable option. But it did show what was happening in practice, and limited the justifications reasonably open to policy-makers going forwards. Supposing that it was thought that section 3 HRA was problematic because it allowed judges to cross the boundary between judicial and legislative acts, policy-makers had to decide:

• Whether judges having a theoretical ability to cross that boundary (not foreclosed by the empirical research) was enough to justify reform;

• Whether judges having gone beyond the boundary in some “exceptional” cases (again, something the empirical research could not foreclose) was enough to justify reform;

• Whether some evidence of wider abuse – the kind which was shown by the empirical research not to exist – would in fact be needed to justify reform.  

The insights from empirical evidence cannot answer these questions. But it can ensure that decision-makers make choices for appropriate reasons. Section 3 could be reformed on the basis that one instance of boundary-crossing is intolerable. But the empirical research meant that it could not be reformed on the basis that there was evidence of this occurring on a widespread level.

All of this is to say that empirical research has a distinct utility, usually deployed in order to evaluate the evidence for a specific and quantifiable claim, rather than to engage with the kind of big picture constitutional issues noted by Jones. Both kinds of inquiry are important: how much so in each case depends entirely on the nature of the question being answered.

Empirical research is a tool which is, in my view, an important and long-neglected one in the context of judicial review and administrative law. But it is just one tool of many. Depending on the circumstances, empirical findings can be vital, important, or totally useless. It should not, and cannot, be used as a substitute for doctrinal constitutional analysis. It certainly provides very little direct insight into the kinds of phenomena that Jones points to – the judicial voice, the reasoning behind cases, the constitutional impact. All of this needs to be acknowledged, and empirical scholars should do well to remember it and factor it into their analyses. Equally, however, empirical work does not need to answer these particular questions: it can answer questions a different set of questions, often those posed by policy-makers. Whilst empirical researchers should not ignore the impact of constitutional landmarks, the judicial voice, important sections of dicta, and so on, those wishing to understand judicial review as a whole should not ignore the insights gleamed from important empirical studies. 

Many thanks to Joe Tomlinson and Lee Marsons for their helpful comments and discussions on this post and its themes in general.

Lewis Graham, Law Society Fellow in Law, Wadham College, University of Oxford. 

(Suggested citation: L.Graham, ‘Empirical work in Judicial Review: A rejoinder’, U.K. Const. L. Blog (13th July 2022) (available at https://ukconstitutionallaw.org/))