Mark Twain contended that there are three kinds of lies: 'lies, damned lies, and statistics'. These matters have become even more vexed since his sardonic remark in 1906. Whilst the (potential) sophistication of quantitative economic analysis has increased considerably, so also have the possibilities of dissimulation via the deployment of sophistical econometrics and other means of quantitative economic bamboozlement. I label these latter groups of wily economic arts collectively as Conometricks. Their common feature is that they all involve the insidious warping and/or specious misuse of (often complex) statistical/quantitative analysis so as to bolster a weak (or dire) economic case, argument, or projection.
A major matter in this regard in the UK concerns the scale and variety of conometricks that have been deployed since 2009 to cobble together some form of ‘economic’ justification for the proposed High Speed 2 (HS2) mega-project. It lacks any commercial rationale.
How, then, to justify an ‘economic’ case for this political vanity project? In 2009 the government set up an offshoot of the Department for Transport (DfT), with staff seconded from the DfT (and from Network Rail, another quango) and called this progeny HS2 Ltd. This organisation was tasked with objectively appraising the ‘business/economic’ case for HS2. Not unsurprisingly, given that HS2 was its only project (and thus its raison d’etre), the new HS2 Ltd quickly divested itself of the tiresome hobble of civil service objectivity. It became instead the dispenser/organiser of a vast raft of government contracts, doled out to a variety of consultancy firms to, effectively, concoct a case for HS2.
On this basis, during 2010/11 both HS2 Ltd and the DfT (which are effectively the same entity) published studies/reports claiming that the estimated ‘Benefit/Cost Ratio’ [BCR] of the HS2 Project was generally of the order of around 2.4 to 2.7 (depending on whether ‘Wider Economic Benefits’ [WEI] were included; or not). All of this sounded quite reasonable on HM Treasury Green Book criteria; which demand a (clearly-identified) BCR above 1.6 (at absolute minimum).
Subsequent evaluations by independent economists revealed, however, that this so-called business case had been built upon a motley variety of extremely questionable assumptions, including:
- Predication of the analysis upon an (assumed) high growth rate for the UK economy for many decades to come (up to 75 years!);
- A numerically high - in the long run clearly fallacious – assumption about how the (supposedly) fast-rising incomes in the UK would convert into extra intercity rail travel;
- Avoidance of the question of what the price of the (proposed) HS2 rail travel in the UK might be (e.g. premium, as with HS1); and, how consumers would react to that;
- Avoidance of the evaluation of alternatives - including both road and rail alternatives to HS2;
- Avoidance of the question about how new developments in ICT might affect the demand for (high-speed) rail travel in the UK over coming decades;
- Avoidance of the matter of the opportunity costs of this (vast) public investment; e.g. as regards other, alternative, forms of communication and connectivity - for example, broadband (let alone roads, or alternative rail).
- Avoidance of the matter, in the cost elements of the BCR calculations, of planning blight; and disruption costs during its (lengthy) gestation.
- The assumption that the time spent by business travellers on intercity trains was of zero productive value, which greatly exaggerated the calculated benefits of higher-speed rail.
In short, the initial HS2 Ltd/DfT ‘economic case’ for HS2 involved a (big) bag of conometricks! The indications are, however, from the independent evaluations noted earlier, that the true/sober BCRs for HS2 could actually be below unity.
What has been the government's response? The DfT (grudgingly) accepted that the HS2 BCRs may be ‘lower’ than it previously proclaimed. However, it is now also trumpets a new line of argumentation about HS2.
We're now told to ignore the previously highlighted arguments (about speed/time savings), and concentrate on ‘capacity and connectivity’. According to the Secretary of State for Transport (11/9/2013), Patrick McLoughlin, transport capacity between the North and South of England will be completely ‘overwhelmed’ within 30 years unless this (very specific; no alternative provision is considered, or evaluated) investment in the HS2 project is made.
The tenor (and content) of Mr McLoughlin’s apocalyptic diagnosis is highly reminiscent of a forecast made in 1894, by a (London) Times writer - working on the basis of ‘precise’ calculations - that London would be literally overwhelmed by horse manure by 1944 (at the very latest).3 Obviously, this forecast did not come about - as alternative ‘connectivities’ to horse- drawn vehicles (e.g. the car, cycles, the bus, rail, tubes, telephones etc.) were developed.
On the very same day as the Secretary of State made his ‘overwhelming-of-England-in-30- years’ prophecy, HS2 Ltd published a new analysis of related matters conducted for them by KPMG (a consultancy firm). This study claimed that HS2 could - by releasing capacity on the old network; and increasing intercity ‘connectivity’ - boost UK GDP by around £15bn p.a. (but only after 2037…something ignored in many media reports).
Whilst the study waffles on about ‘forecasting the future’, it also makes plain, in an upfront disclosure (p.ii), that its analysis does '...not constitute a forecast' (emphasis added). We have here, then, a classic example of a Not-A-Forecast Forecast; or a NAFF, as I call them for short. [NAFFs are actually quite common: they are otherwise known as ‘deniable’ forecasts: the forecasters involved deny responsibility for their own forecast).
Like its predecessors, the KPMG study also neglects alternative means of achieving ‘connectivity’ (e.g. ICT; roads; other rail); and fails even to compare and evaluate its preferred ‘solution’ against alternative possible rail improvements. And buried in its Technical Appendix (at 6.3.38; p.83; if you can stick with it that far!) the study accepts that a key element of its approach 'does not have a firm statistical foundation’.
The real conclusion is that the KPMG 'technique' employed ‘captures Sweet Fanny Adams’. Indeed Professor Henry Overman of the LSE (and a sometime panel adviser to HS2 Ltd, until June 2012) has suggested that key aspects of the KPMG methodology are ‘technically wrong’ and that, in his own analyses, properly accounting for the effect of other variables had reduced the impact of rail connectivity by six to eight times.
Despite all of this we are told that ‘Ministers are due to publish an updated case for the (HS2) line, drawing on a variety of new work; including the KPMG report’ (emphasis added). It therefore looks like we’re in for another round of taxpayer-financed, pro-HS2 shenanigans coming along soon. A government spokesman has said that ‘the case for HS2 will be improved further, when we publish shortly the updated strategic case…’
The (all-party) Treasury Select Committee (TSC) has, however, (rightly) expressed serious concerns about the quality of the BCR analysis conducted to date to support HS2; and called for the Treasury itself to evaluate the claims put forward by HS2 Ltd and the DFT. I would go even further: I think the UK Statistics Authority should take a good, hard, look at the ‘conometricks’ that have been deployed to support HS2 - at very considerable public expense.