“Why economists tend to be in bad odour …” – Cycleways in Christchurch

In an 1879 essay, Francis Walker tried to explain “why economists tend to be in bad odour amongst real people.” Walker, who went on to become the first president of the American Economic Association, argued that it was partly because economists disregard “…the customs and beliefs that tie individuals to their occupations and locations and lead them to act in ways contrary to the predictions of economic theory.” – Frank et al  1999

As some of you may be aware, Christchurch City Council has applied for NZTA funding to develop the network of major cycleways illustrated below.

This economic appraisal of this investment is discussed in detail in this blog post. The benefit-cost ratio for the $160 million investment was estimated using NZTA’s Economic Evaluation Manual to be 7:1, with the benefits of the project breaking down as follows.

All well and good.

Until about a week ago when two economists from the University of Canterbury, namely Glenn Boyle and James Hill, released their review of the business case for the major cycleways network. Based on their review, Boyle and Hill conclude the actual BCR is more likely to range from 0.7 – 1.6. They reached this conclusion for the following reasons:

1. Fuel prices – Rather than the 40% increase in real fuel prices assumed in the business case, Boyle and Hill suggest a more reasonable assumption is constant fuel prices. This reduces the BCR to 6.0.
2. Time savings – Boyle and Hill calculate the average time saving per vehicle trip and conclude that because it less than 6 seconds, that these benefits should be discounted. This reduces the BCR to 4.6.
3. Safety benefits – Boyle and Hill argue that the procedure used to calculate safety benefits is designed only for small projects costing less than $5 million. Removing these benefits reduces the BCR to 3.8
4. Health and environmental benefits – Boyle and Hill argue that the procedure used to calculate health and environmental benefits is designed only for small projects costing less than $5 million. Re-calculating these benefits reduces the BCR to 0.98 – 2.3
5. Discount rate – the business cases uses a 6% discount rate; Boyle and Hill argue “market realities suggest this is probably too low”. Using a discount rate of 8% reduces the benefits to 0.7-1.6.

Oh dear.

I’ve subsequently reviewed Boyle and Hill’s review. My general conclusion is that while they make some valid points, they miss the mark in the places that matter. This in turn means that their conclusions are at best unsubstantiated and at worst simply wrong.

The key issues I find with their analysis are summarised below.

The first issue relates to their grounds for dismissing time savings. First, Boyle and Hill the divide total (estimated) time savings by the total (forecast) number of vehicle trips in Christchurch so as to calculate the average time saving per vehicle trip. They then reason that because the average time saving per trip is ~6 seconds, then the time saving benefits are too trivial to be included in the business case.

This analysis smacks of the sort of erroneous logical reasoning that one would critique in a first year statistics course.

Consider the statistical distribution of time savings that might result from cycle investment. It’s reasonable to suggest these savings will not be distributed evenly.

More specifically, the time savings will accrue primarily to vehicle trips that occur within specific corridors and at specific times. That’s certainly what the modelling of cycle flows seems to indicate, as shown below. From this map one might expect very small time savings for vehicle trips in areas such as the airport, Brighton, and Banks Peninsula, with larger time savings for vehicle trips travelling in the peak direction on key radial corridors.

The need to consider the distribution of time saving can be illustrated with a stylised example. Imagine a city where there are 1,000 vehicle trips, and where a proposed cycle investment will save 2 minutes for 50 vehicle trips, while the remaining 950 vehicles trips are unaffected. The average time saving in this city is only 6 seconds per vehicle trip, even if every vehicle trip affected by the investment actually saves 2 minutes.

The takeaway message is that the localised time savings in a large transport network cannot be accurately represented by the average time saving per vehicle trip. The latter metric may indeed obscure what are tangible savings for a small number of vehicle trips.

Or, to put it another way, if Boyle and Hill wanted an even more sensational figure, then they could have averaged time savings over all vehicle trips in New Zealand rather than just Christchurch, and concluded that the project would save less than half a second per trip. But that would be even more ridiculous.

Reductio ad absurdum; QED Boyle and Hill’s dismissal of time savings is unsubstantiated (NB: One could of course analyse the distribution of time savings and consider only those savings that were above a certain minimum threshold, but Boyle and Hill have not done this).

The second issue relates to their choice of an alternative discount rate. The major cycleways project is a transport investment which is seeking funding from NZTA’s land transport programme.

Let’s make this very clear: NZTA specify that a discount rate of 6% is to be used when undertaking economic appraisals of transport investments. It is therefore entirely appropriate for the business case to use a 6% discount rate. If Boyle and Hill have an issue with the discount rate that has been chosen by the NZTA, then they should raise those arguments in an appropriate forum – not pretend it’s an issue associated with the major cycleways project.

At this point it’s worth pausing for a moment and simply noting that if we add back in even 50% of the travel-time savings and use the 6% discount rate stipulated by NZTA, then the BCR for the cycleways project is likely to return to respectability – even if we accept all of their other points. And I don’t …

The third issue relates to their discussion of travel behaviour change. Boyle and Hill question the magnitude of travel behaviour change, but ignore that the business case takes a conservative view of behaviour change when compared with the results of stated preference surveys.

Stated preference surveys in Christchurch suggest that up to 30% of people would be willing to switch to cycling if sufficient safe infrastructure was provided. At present, the Census suggests that around 7% of people in Christchurch cycle to work, while the Household Travel Survey suggests that around 3% of total trips in the Canterbury region are made by bike.

The modelling did not conclude that cycle mode share would increase to 30%. It took a much more conservative view, which is that the cycleways would boost the number of cycling trips by 15-35%, which would imply an increase in mode share of 1-3% to between 5-10%.

A bit of international context would help here. Other cities that have invested in transformative changes to cycling infrastructure have experienced much larger increases in cycling mode share. For example, the Dutch only started investing in safe, quality cycling infrastructure in the 1970s. Today, many Dutch (and Danish and Swedish) cities have cycle mode shares in the range of 20% to 40%. Moreover, more modern cities like Portland have achieved cycling mode shares approaching 10%. QED the travel behaviour change assumptions in the modelling are within the range of what we observe elsewhere.

The fourth issue is that Boyle and Hill have misread the EEM guidance on analysing cycling benefits. They claim that cycling benefits have been estimated using an inappropriate (“simplified”) procedure. However, that’s simply not true. The values used to calculate per-km benefits for cycling are part of the core EEM (Appendix A20, if anyone’s interested). QED it was appropriate for the original business case to include these benefits.

The fifth issue is that Boyle and Hill make non-standard assumptions about fuel prices. They criticise the modelling for assuming that real fuel prices will increase. However, the assumptions in the business case are based on modelling published by MBIE – who are hardly a bunch of peak-oil alarmists. Boyle and Hill’s critique basically boils down to “oil prices are around their historical norm”, therefore we should not assume any future increase in price.

As economists, they should know that past performance may not be a good guide to the future.

Boyle and Hill argue that crude oil futures quotes are expecting current prices will persist, although I understand these contracts 1) typically extend out only for the next decade or so, and 2) the liquidity is fairly low in more distant future years. In contrast, NZTA stipulates a 30 year evaluation period for similar transport projects. Moreover, unless Boyle and Hill have an alternative (forward-looking) model of fuel prices, as well as evidence that their model is more accurate than MBIE’s, then their objection to the fuel price assumptions used in the business case is somewhat vacuous.

If, as it seems, Boyle and Hill’s real target is NZTA’s evaluation methods, then their critique of the cycleways is at the very least misplaced. Christchurch is proposing to invest a decent amount in cycleways, but that expenditure is dwarfed by state highway spending. If, for example, Boyle and Hill applied the same attention to the $11 billion RoNS’ programme, then they’d find some projects which start out with BCRs less than 1.0, even with more uncertain agglomeration / wider economic impact benefits included from the outset.

A serious investigation would have at least considered this wider transport investment context, before honing in on the cycleways project as perhaps a case study. And even then it’s a relatively non-representative (and unimportant) choice of case study.

So where does this leave us?

Well, if the aim of Boyle and Hill was to create clickbait for anti-cycling neanderthals, then they can rest happy in the knowledge that they have done their job exceptionally well.

However, if they wanted to foster more informed public debate on the merits of the major cycleways project, or the business cases for transport investment in general, then they have clearly and demonstrably failed. It’s especially unfortunate their review comes across as a deliberately sensationalist hatchet job with largely unsubstantiated and/or incorrect conclusions. By extension, their review does not – as they claim – call into doubt the key finding of the original business case, i.e. the investment in major cycleways represents a relatively effective transport investment. For these reason I wouldn’t expect their review to hold much sway with Christchurch City Council and/or the NZTA.

Postscript: While Boyle and Hill’s review is, in my opinion, “in bad odour”, it is encouraging that the business cases for transport investment are receiving more attention from the wider economics profession in New Zealand. I’d certainly encourage Boyle and Hill to pursue this new found interest further, and would welcome them scrutinising the business cases for RoNS projects, many of which cost in the billions of dollars and start with BCRs less than 2.0. That’s really where the real money is being spent, and it’s where the economic evidence is the weakest.

Share this