A few weeks ago, Steven Joyce spoke regarding traditional cost-benefit basis, here is what he said reported in NZ Herald:
Some projects in Auckland, such as the City Rail Link, did not stack up on a traditional cost-benefit basis, he said, although the Government has committed to funding for it.”
“I think there is unfinished business now for all of us to think about what are the true wider benefits of some of these projects and trying to get a bit more discipline to them in the years ahead,” he told the finance and expenditure committee at Parliament in response to questions from Greens co-leader James Shaw.”
“From my perspective, I think it is important that we go through the benefit-cost ratio discussion.”
“However I would signal that some of the projects that we collectively have all committed to, including the CRL, doesn’t really stack up on a traditional cost-benefit basis.”
Joyce is off the mark, however, the issue is not that we need better discipline but if whether the current way we calculate for business cases is even fit for purpose for the 21st century. It is telling that, after tens of billions of dollars spent on building roads over the past decade, congestion is now worse than ever. This suggests the current approach to prioritising really isn’t working and isn’t achieving its desired goals.
In times of discontinuity extrapolative models fail
One of the major issues current business cases have is they are heavily orientated around modelling. While modelling is a very useful tool we must also understand its weaknesses, one of the large weaknesses is inputs are based on historical data. This is great when we are moving along business as usual however in times of discontinuity these models fail. As we have seen, traditional modelling has on many occasions greatly under forecasted public transport demand. For example, recent work on improving rapid transit to the North Shore found the Busway is likely to start hitting constraints around a decade earlier than originally expected and could be much sooner if growth continues in the way it has recently.
Another classic example is the original business case for building Britomart in which just over 21k trips per day were expected to use the station by 2021, as of last year the number was at 42k.
This modelling under forecasting has wide implications for Auckland for two reasons:
- Lower passenger demand forecasts dramatically affect the benefit-cost ratios of PT projects as it understates the benefits it will provide.
- Under modelled forecasts result in PT step changes or implementation being funded to late.
Wider Economic Benefits are Sidelined
When discussing wider economic benefits, traditional engineers remark on how they are potentially subjective/hard to calculate & really should not be added or prioritised in business cases. However, this thinking is incorrect as we don’t build transport projects just for transport we build them because they enable aims like more housing, more economic growth, better environmental outcomes or a better city. Transport is always a means not an end. This type of thinking prejudices public transport/active modes whose real benefits go beyond just transport but enabling development as well as environmental/health outcomes. Also, systematically under-estimating wider economic impacts is like to lower the priority we give to projects that bring people together and maximise agglomeration (like the CRL) rather than those which encourage economic activity to disperse (like Puhoi-Warkworth).
For example, the City Rail Link business case had to account for the costs of all the land purchases but is not allowed to account for the sale/redevelopment of that land.
The Models are not as Scientific as people think
When people discuss models many people assume they are like they are in the natural sciences, however in transport this isn’t true as models in transport have many subjective assumptions built in which creates bias/perverse outcomes. For example, many models have assumptions such as traffic cannot fall even though the abundance of international empirical evidence of traffic evaporation & the Braess’s paradox such as Cheonggyecheon, Embarcadero Freeway, Utrecht, Paris, or even closer to home in Auckland with even the large amount of CRL works disruption. Congestion is lower than before the works on most streets.
This has a large effect on what we can do with the City Centre if traffic is assumed to not be able to fall or can only fall slightly then when asked can we re-prioritise street space the computer will always say no. Other assumptions such as large transfer penalties also effect business cases, resulting in lower modelled PT demand, even though we know given the right conditions people will transfer. Travel times normally include walking/cycling to and from PT stops, but not time spent walking to or from a car, or the time spent even finding a car park. Many assumptions are very predictive, for example the 2011 AWHC Business Case and the Ministry of Transport’s review of the first CRL business case both assumed parking prices in the CBD in the 2040s at rates lower than they are today.
Up until recently we assessed the benefits of infrastructure projects over just 30 years. This was not in line with other countries such as Australia who were using 40 years, which we have now moved to, or the UK which uses 60 years. This is important as imagine the increase in benefits in the business case for example if the CRL was assumed at 60 years of benefits rather than 30/40 years. For example, the Metropolitan line in London was completed 1863 and the original New York Subway Line completed 1904. While these have upgraded over the years, this infrastructure still in use today and more valuable than ever. While I am not suggesting we use 150-year timeframes it begs the question are we truly evaluating multi-generational infrastructure projects like the CRL.
Is a lack of diversity behind some of these assumptions?
The transport field is unfortunately not a very diverse one, this is a problem as diversity brings different perspectives/ideas. People plan what they know and if the field is heavily skewed towards a certain demographic, are we really surprised to find model assumptions that really only focus on peak commuting to work.
For example data from the US shows that commuting only makes up 20% of trips & a much smaller amount for women. While the data shows men travel more distance, women make more trips. Designing models simply around just peak commuting makes little sense, what outcomes would we get if instead, we focused modelling/designing for trips instead of the commute?
Sticking to discipline is pointless if the manual is not fit for purpose for the 21st century. What is more important is we realign our assumptions to international best practice.
Processing...
Excellent post, Harriet. This is the nub of the problem. We need a CEO at AT who can cut through the false assumptions and stuck-in-the-mud thinking.
How does the CRL not stack up on a traditional cost-benefit basis?
I cannot find figures anywhere that form the basis of this assertion from Mr Joyce. Anyone?
The MoT produced a hatchet job business case using traditional methods and traditional (everyone prefers cars, no one will transfer, no one will live in anything other than a quarter acre McMansion) assumptions.
Yes. There has never been anything wrong with the methods themselves. It is the one-eyed assumptions of the people doing the input. It is a fairly straight forward process to update a mode split model based on revealed preference from the Northern busway and the post DMU data. If we don’t do that and don’t model then all you have is somebody’s assumed result. Depending on who they are you probably won’t agree with them.
+1, I think part of the problem is that we have really accurate traffic flow models for intersections where the assumptions that we make have very limited impact. The same people then try to model public transport patronage or induced traffic. They take the assumption that ‘if your model changes by 20% then it’s a good sensitivity test’ rather than realising that assuming people will take a 20% or 0% time and cost penalty for a car over PT might change your patronage by a factor of 5, but are still appropriate sensitivity tests.
Because the assumptions in an intersection model are trivial we seem to expand that to macro models rather than even bothering to calibrate against the observed values.
Kind word for transport modelers; if your model doesn’t predict the present, you can’t believe it will predict the future.
Every project assessment starts with output from the region-wide land use and vehicle/PT demand modelling undertaken by Council. For any project, these outputs typically set your boundary conditions and then are refined down to the micro-simulation level to understand local traffic effects. All of the big projects – Waterview, EWL, AWHC, CRL would have used these strategic models as a starting point so I’d encourage anyone concerned about future transport projects to request those base year and future year outputs (available from Council) and take a look at what changes (vehicle volumes/ PT patronage) are anticipated over time.
“For any project, these outputs typically set your boundary conditions and then are refined down to the micro-simulation level to understand local traffic effects”
And that, dear reader, is how the SH20 motorway came to have traffic lights with stationary traffic on it every weekday.
Did you even read my comment before responding to it?
The problem is that some projects are large enough to change the outcome of this model, people either do not go back to the model, or the model does not factor in the change the project would create. Insanely, these models often don’t even factor in that driving and PT compete for users.
Missing Link, yes, you’re right, that is the starting point. The Council model assumes the traffic increase will happen anyway, regardless of the project being modelled.
Have a look at the model structure in the traffic modelling report for Waterview. There is no flowpath from the project improvements box to the regional traffic flow box! The ART3 model predicts the increase in person-trips, but assumes that the Waterview Connection will have no influence on it. This is absolutely flawed. If the model doesn’t show that the roading projects are creating the traffic, and keeps showing incorrectly that the roading projects will ease congestion, guess what? You build roads.
Instead of working out the person-trips as if they are something the planners have no control over, they would do better to calculate traffic increases simply by calculating the increase in roading capacity, since the fundamental law of congestion links the two.
One day, I hope the calculations will be to work out the traffic decrease by calculating the decrease in roading capacity.
Yeah, I read it and there is a feedback loop where the project is added in and demands redistributed. That’s why Waterview connection is estimated to have a pile of traffic on it and local roads have a reduction. But the origins and destinations of people are unchanged – the residential and employment zones used in the model are fixed by the District Plan, Unitary Plan etc (Council land use policy). The model simply assigns people along the quickest route between those land uses.
The feedback loop doesn’t feed back into the calculation of the regional trips, though. It’s a redistribution of the person trips, which both the text and figures show are kept constant. With the person trips the same, and the motorway traffic increased (as it has more capacity) of course the local traffic goes down. The model doesn’t allow for anything else. See the figures in table 5.3, and the explanatory text:
“Between the Do-Minimum and the option, in both 2016 and 2026, there is only a very small difference in total trips (less than 1%). This is because these totals are at a regional level, and the project does not add many trips at a regional level, but does alter patterns at the local level…”
The note a little further on explains why there is any change in traffic numbers at all between the Do Min and the Option:
“The predicted change between the do-minimum and option scenarios include the effects of changes in distribution (destination choice), mode choice and time of travel (peak spreading).”
The really big changes, of traffic induced by the roading project itself, have not been modelled.
The project “not adding many trips at a regional level” follows hot on the heels of all the other motorway projects that have “not added many trips at a regional level” and of course, they all have. That’s why we have so much traffic.
This is a conventional traffic model which has been debunked by many researchers. Todd Litman (Victoria Transport Institute) details what they should do to modify this model to allow for the subsequent increase in traffic.
I wonder whether the ART3 model is sensitive enough to be run twice, once with the WRR, once without, to show the real difference.
I’d go even further and suggest many of these “one-eyed assumptions” are the result of systemic bias. Engineers are taught from day 1 to never be non-conservative with their assumptions. This is obviously a good thing when designing buildings and dams, but it’s much less obvious when designing transport networks. When dealing with traffic, being “conservative” is often interpreted as making sure traffic growth assumptions are not too low. Engineers get grilled if they design an intersection that ends up not working well because their traffic growth assumptions were too low, whereas they don’t get grilled when they over-engineer an intersection through using conservative traffic growth assumptions. So I suspect there’s a tendency among some engineers to lean towards traffic growth assumptions that are on the high side rather than the low side. If actual traffic ends up being lower than predicted, then that’s a bonus.
But with public transport it’s the complete opposite. Because funding is much tighter and more directly dependent on user numbers, being “conservative” means assuming lower numbers of users. Engineers get grilled when a new PT system is under-utilised as that’s seen as a failure, whereas engineers get praised for designing an over-used PT system as that’s seen as a success. Therefore it’s rational for an engineer to lean towards assumptions which predict lower PT user numbers over assumptions that predict higher PT user numbers.
I think this bias would disappear, or at least reduce, if we had direct user-pays system for drivers, like we do for PT (e.g. congestion charging), as this would remove one of the fundamental differences in the way that PT projects and traffic projects are assessed.
“Engineers are taught from day 1 to never be non-conservative with their assumptions” – Sometimes only when doing basic or high level analysis.
My own engineering background is quite diverse (embedded design being a very broad field), however when engineering an electronic circuit or some types of software, conservatism can fly out the window. This is due to pressure to reduce the unit price (for a circuit), meet deadlines or fit within tight environment constraints (software for embedded design often runs on a low power, low memory cpu/mcu).
What this means in practice is that when designing a circuit with a 12m warranty, you may use a (for example) capacitor rated to only 4000 hrs life – 2h15m use per day rather than an 8000hr lifespan cap. You’d only be saving 8c each in 200x quantities, but that adds up very quickly in a complete circuit.
🙂
Yea fair point Jon, engineer is a very broad term. I was really thinking of civil engineers who design large structures that would kill large numbers of people if they were to fail. I think that risk-averse civil engineering culture is the environment a lot of traffic engineers were trained in.
“I was really thinking of civil engineers” – Yeah, I get that 🙂
Perhaps I was a bit sensitive. Every career I’ve pursued has been vulnerable to misinterpretation.
RE: Risk aversion – I don’t feel that the aversion to risk is itself the issue, rather the issue is failure to update models and methods to be a better fit with empirical evidence. To me this is incomprehensible.
Great post. I would offer several further causes:
1. The models have evolved to match business cases, not reality. We now model future scenarios with demographic scenarios that are not modified by the infrastructure choice, because that allows “apples with apples” comparisons of benefits for BCRs. Yet we know that land use – transport interaction is one of the primary ways that the “spare capacity” of new roads is filled in.
2. Our models cannot cope with changing behavioural norms. They assume that demographics and infrastructure may change over time, but behaviour remains constant. We have policies designed to change travel behaviour, so this makes no sense.
3. Pressure to “get projects moving” biases results. We do lots of analysis, but often it is after the decision to proceed with a project has already been made. Then it is too late for analysis. The pressure is to get the project moving. We really need to have more independent analysis before a commitment has made. I know from experience that modelling results that do not tell NZTA what they want to hear get far more scrutiny than results that it the project narrative.
The only thing harder than getting a minister to do what is needed, is to get him to stop doing something futile, once he has promised it to voters.
Many of our models (both employment and demographics as well as transport) are based on old structures, dating back to the 1960s or earlier (read up Wardrop). Things that were good ideas then may not still work. Spain spent a fortune on infrastructure before the GFC, but still went into recession. I hope we do not follow suite.
+1 exactly
As a side note, models in natural sciences are not at all different in the sense that they are only models. It’s the scientist’s job to know the limitations and to correctly interpret any results. Same for transport models.
I wonder if the science is so indeterminate and factors hard to predict that we may as well just have a bunch of committees that brainstorm ideas and then the public vote on them after design & costings done. When you start talking about 30, 40 or 60 year periods & various inputs we determine, seems impossible anyway. Should be done from a more philosophical angle.
Well this approach wouldn’t have been worse than what we’ve had, anyway. But in the case of traffic modelling, the science is way further ahead than what the engineers NZTA appoints are using. If they had used best practice, Waterview Connection would never have been built, and the East West Link would have been thrown out the window long ago.
The model structure for these roading projects is overtly, fundamentally flawed. The engineers wouldn’t speak to me directly about this, so I have passed some questions to them via NZTA. I have been waiting for a response for two weeks; hopefully I’ll receive something soon.
“If they had used best practice, Waterview Connection would never have been built, and the East West Link would have been thrown out the window long ago.”
The decision to build Waterview was political, not technical. It was the result of explicit government strategy. If the engineers concerned had resorted to “best practice” they would have been looking for alternative employment.
+1, and if the government had used best practice transport planning then Waterview would have been built with the motorways stopping at Greenlane and St Lukes.
“Best practice” is a very subjective term. The best solution to an engineering problem is a function of how the question or scope is framed and the relative weight attached to what can be conflicting objectives.
Completely agree, though this is slightly different becuase transport planning isn’t an engineering problem, it’s a planning problem.
The best solution to a planning problem is a function of how the question or scope is framed and the relative weight attached to what can be conflicting objectives.
And within any feasible definition of scope and weighting of objectives, best practice would not include moats cutting to or through the CBD.
The issue isn’t Waterview connecting with SH16, it’s Waterview having east-connecting ramps to the CBD.
“The decision to build Waterview was political, not technical. It was the result of explicit government strategy.”
Yes. Still, the traffic engineers need to be taken to task. When the traffic forecasts erroneously show 50,000 cars taken off the local roads daily, it makes all sorts of political decision-making difficult. At government level, how does a progressive politician discuss PT alternatives, short of having to debunk the whole model?
At a local level, the level of funding put into safety measures is way lower than if a correct forecast was produced – that most of the easing of congestion will have dissipated within 5 years, and all by 10 years. If I was a safety engineer at AT or NZTA, I’d be pissed off with the engineers modelling the traffic flows, as it undermines all the work they are doing to make our roads safer.
I know an engineer working for NZTA. He tells me that it’s sometimes disheartening that they’re not able to do what they think is right, due to their managers sometimes pressing a certain path.
This isn’t to say that it’s always the case that they’re pressured, only that it does sometimes happen.
I think that it’d be more helpful if the mainstream media was to find out if there was political interference (internal or external) and kick up a storm when there is. Be an improvement on stories about lost pets, etc.
This is a really interesting post and one I congratulate Harriett on producing.
I tend to agree with much of what is said. The elephant in the room with transport projects is driverless cars. This technology is going to revolutionise the economics of vehicle transport. The commonly accepted 1.2 people per car is going to be more like 2.5-3.0 per car. Driverless cars will work like a ride-share scheme via a centralised system or app, bringing people together like never before during busy times.
I wonder how many urban roading projects are taking driverless cars into account during cost/benefit analysis?
Ride sharing in cars doesn’t really happen now, family members excepted, so why do we think autonomous cars will change this? I haven’t seen a good case on the ‘why’ yet.
Good question I believe the answer is they acknowledge the potential of AV’s, CV’s and MaaS but apart from ATAP which goes a little further than most http://www.transport.govt.nz/assets/Uploads/Land/Documents/ATAP-Technology-Report.pdf really don’t apply it.
The real questions are
1. Will MaaS, CV’s & AV’s deliver the above, competing studies on this.
2. Even if 1. will this tech be delivered within a reasonable time frame, again competing studies.
3. Though not related to your comment on urban roading projects is will AV’s CV’s & MaaS tech be also applicable to PT i.e. Driverless at grade Light Rail (Most likely) & also will AV’s CV’s & MaaS be complementary to to PT rather than competitive.
+1, I never understand the ridesharing will destroy PT argument. PT is rideshare, and it’s far cheaper than ridesharing in a car because 1 $800k vehicle can carry 100 people at a far lower fuel cost. If we also assume that road pricing is introduced then there is a massive financial incentive to share the cost.
That, plus I don’t think we will ever have driverless cars that operate in all environments.
Coming up with an accurate time frame is the big issue, especially when applying a discounted methodology as a difference of 5 years can make a material difference. It’s a dart throwing exercise which is why in valuations we often see a best case scenario and a worst case scenario.
Indeed I do believe driverless technology will also effect light rail, bus and heavy rail as well. However from a cost/benefit perspective I believe we will see a larger effect on road vehicles.
Nick – there is absolutely no evidence that driverless cars are likely to have any effect on number of people per vehicle. Certainly, if anything, I would think it will go down rather than up. Why would a driverless car stop to pick up someone else? Are you presuming that people would voluntarily want to split their fare with some random other person they have never met, who may or may not be nice, may or may not smell good, may or may not offer to pay half the fare? Complete balderdash.
Its going to revolutionise things alright. Probably going to cause an almighty great traffic jam of loose unit doodle cars with no drivers and no parking space, crawling round city blocks in electric mode first gear.
at a minimum driverless cars problematise the ‘value of time’, which is a model assumption that would need (re) consideration.
Why would it revolutionised the economics? If people don’t rideshare now when the costs are high, why would they start ridesharing when it becomes cheaper? That makes no sense.
The time horizon for driverless vehicles and associated BCR needs to reflect potential changes in ownership models.
My view is that we will start to see the driverless technology components applied during specific sections of the journey i.e. on a motorway/expressway first and then once the issues have been resolved, start to see the technology applied in other places. This effectively chunks the problem to one step at a time, allwoing the solution to a specific aspect of the problem to be resolved, then the next one started.
Also what would happen if driverless vehicles were banned from the CBD (ringed by the present motorways)? would the utility of the vehicle be compromised?
Driverless buses I see as the only really useful innovation. They are already being tested and once practical will make small buses economical. Currently they are not as the driver is the biggest cost so a smaller bus makes no sense.
I agree with people above, why would people suddenly be willing to jump in a shared car when it is driverless when they aren’t now? Small mini van buses may be more desirable, probably to the nearest rapid transit link with a driverless train.
And driverless trains (like the Canada line in Vancouver) and trams will become really attractive.
Can someone please explain why driverless cars wouldn’t just increase congestion as rideshare services like Uber have done? Quote below from StreetsBlogNYC
“It’s Settled: Uber Is Making NYC Gridlock Worse
“The controversy over Uber’s impact on Manhattan traffic has been settled. Uber, Lyft, and other app-based ride services are unequivocally worsening gridlock in the Manhattan core as well as northern Manhattan and the western parts of Queens and Brooklyn, according to a report released today by transportation analyst Bruce Schaller.
“The new ride services, known as transportation network companies, or TNC’s, last year caused a net increase of 600 million vehicle miles traveled in the five boroughs — a 3 to 4 percent jump in citywide traffic, Schaller found. This trend marks a troubling inflection point — for the first time in many years, car-based services, not transit, account for most growth in travel.”
Driverless cars will likely see an increase in ride sharing, However I would expect that those who can afford it will have their own private cars. This will potentially create a situation where a car will take you to work and then drive home empty (because there is no parking) and then then will drive back at lunch time so you can run some errand go out to lunch and then drive home again and then come and pick you up at the end of the day. So we will likely see significant numbers of cars driving around with 0 passengers/people.
How all this will play out is anyone’s guess, but I wouldn’t be too quick to assume we will see a significant increase in the average numbers of people per car.
Thanks, that’s what I thought. It seems to me that in the cbd especially, driverless cars will simply increase traffic, for the reasons you gave. Maybe elsewhere, too.
There are a couple of things that need disentangling. And forgive me for what seem like waffle, but I have a point 🙂
First, defining goals is a political process (used in its broadest sense) – there is no technocratic way to arrive at goals (even things like water quality standards, which seem scientific, are political). So, for any business case/project etc, developing goals will be a political process. Now, we can validly criticise whether they are sufficiently explicit, but we can’t criticise them for being “wrong” except in a political sense
For example, let’s say we have a business case for a new tank that needs a new gun that can penetrate 500mm of RHA at 2500m. The only grounds we have to criticise this goal are political – “why do we need such a beast?” or “that tank won’t cope with T-80Us!”, but not scientific. And, at times, MAXIMISING BENEFITS TO COSTS MAY NOT BE THE GOAL OF THE PROJECT!!! (i.e. for most defence projects the expected payout is negative, except if there’s a war).
Second, identifying which option best meets the goal is a scientific process. We might have 3 tanks, and they all have different guns, and different penetration curves. Maybe they all do 500mm at 2500m, but there are various scientific aspects to consider – this gun doesn’t have a chromium lined barrel, this one is smoothbore etc etc. This is where the technocrats rule: identifying the best technical solution to a political goal that is expressed in technical terms.
Now, here’s where road projects come in. The flaw is this: the goals developed by the political process are not explicit enough, not teased out enough. But this isn’t a technocratic debate, it’s a political debate. Using the CRL as an example:
WHAT ARE THE GOALS?
What generally happens is that we rush to solution design before we’ve truly explored, debated, discussed, and confirmed our goals. So rather than a logical flow, we get a constant, messy interaction where solutions pop up in their various garbage cans, where goals are constantly redefined, where the political and the technical are mixed and confused.
The CRL to me can best be conceptualised as two separate, sequential projects.
#1 to me is the Broad Option Project, the *political* goal being to achieve the most efficient movement of people to/from CBD etc.
Then the technocrats do their work, and say “A train loop”
#2 is then the CRL Project itself, with a *political* goal of the train system that can move X people at Y times
Then the technocrats do their work and suggest Option A, Option B, with 3 stations, with 4 stations, with 5 stations etc.
What we have, however, is that technocrats meddle in politics, and politicians meddle with technocrats. It’s the goal of the politicians to set the goal – a library with customer satisfaction of 90%. It’s not up to the politicians to tell the technocrats what colour to paint it or what books to stock it with.
I prefer my tanks filled with water personally.
A lot of truth here, and it’s just missing the words ‘gravy train’.
“The only grounds we have to criticise this goal are political”
That’s not a goal. You’ve described a solution.
The goal would be to “Have a defence force that can defend New Zealand against our 5 main threats and participate in peacekeeping missions simultaneously”. – This can be politically criticised.
The problem is then “our tanks cannot defend against key parts of the army of country x, Invasion by country x is a top 5 threat”. – This can be technically criticised; as the basis on which the threat is put in the top 5 may be flawed, or the finding that we can’t defend may be flawed.
The solution is then to build “a new tank that needs a new gun that can penetrate 500mm of RHA at 2500m”. – This can be technically criticised as we may be better to counteract the tanks in other ways, or to prevent the tanks landing, or other alternatives, or achieving the solution may cost so much that it threatens our strategy.
Those criticisms *are* technical; they discuss how the technical implications at each stage impact the strategic goals.
In terms of transport:
The goal might be to “increase access to employment and social opportunities and improve freight access to businesses and ports”.
The problem may then be “congestion in south-west Auckland reduces public access to employment in the airport-Onehunga-city centre corridor”.
The solution may then be “a new rail line that can move 5,000 additional people each way on the corridor”
Oh, and the two projects that you describe at the bottom are usually done in four stages of increasing detail.
A gun that can penetrate 500mm RHA at 2.5km isn’t a solution, it’s a goal (requirement). It’s generic, outcomes focused (the outcome being a brewed up early model T-72, but be careful of the -72BMs!), and is open to multiple solutions. A Rheinemetall L44 or a Petrov 2A45m is a solution. Now, we have to realise we’re dealing within boundaries, and you’re right in that we might have a “higher level” goal of “ensuring survivability against threats out to a 2500m windows”, or which a sub-goal is this particular gun. But a particular required capability is never a solution as long as it isn’t technically specified.
Back to CRL, let’s say the strategic goal is simple – “maximise overall speed of movement across Auckland”.
Now, within that there are a number of sub-goals, probably geographically rather than mode bounded to avoid solution capture. Let’s say we have a goal of maximising overall speed of movement from West Auckland to the CBD.
Now we have a *technical process* of identifying the best solution to this goal. This is not a political process. However, once this solution has been identified, it is then transformed into a new political goal “The most efficient West-CBD train link”, which is then technocratised, politicised, and so on. So you are right in that it isn’t a simple political -> technical, but rather that (ideally) the phases are sequential and clearly separated.
If the Minister of Health sets a goal of reducing cancer in NZ by 50%, then the only valid non-political criticism is what mechanism (primary care vs. surgery etc) is most efficient in achieving that goal. The goal itself is sacrosanct at a technical level.
We had this sorted in the 1980s. Outcomes based thinking. It is the goal of the politicians to identify the desired outcomes of the public service. Public servants (technocrats) then negotiate with the ministers the delivery of outputs to do that, with a bargaining process. But politicians don’t meddle after that.
We could do it now. All it would take would be Phil Goff saying to Auckland Transport “I want average commute times in 2020 to be 10% shorter than now. Go forth and do it.” Then it’d be super, super easy to criticise business cases, because it’d be clear what they’re designed to achieve
Yeah you’re really bad at this goal setting business, and your approach to BCRs suggests you don’t understand them at all.
Your method is to describe an outcome in very precise terms, call it a goal, then what, get that outcome? You don’t need a cost benefit analysis for that, you just need to go buy the thing you’ve already decided you want.
For example, setting the goal of the CRL to move X people in Y time is an extremely bad way to start a cost benefit analysis that ignores why you would do one in the first place. What if you randomly determine it has to move 5,000 people an hour, but there is one solution that moves 4,900 people an hour but is twice as fast and saves a billion dollars on construction? If you have pre-decided your solution by calling a technical specification a goal, your wasting your time and certain to get a terrible outcome. This is why motorway BCRs are shoddy, they go in already decided that a motorway is the answer, when an expressway might have done 90% of the benefits for 60% of the construction costs and be delivered five years earlier. You seem to want the same thing for your tank guns.
A CBA is basically a multivariate analysis taking into account dozens of factors in the evaluation. The goals should be multivariate and imprecise so that you can actually discriminate between options.
The goal for the CRL should be something like “what is the best way to maximise the effectiveness of the existing rail network”.Then you evaluate across travel times, capacity, station locations and catchments, construction cost, operating cost, land development potential, land value capture potential, delivery risk, etc etc.
By your logic an intersection that can throughput 5,000 vehicles an hour with average delay of less than 10 seconds is a goal not a solution and is therefore not subject to technical criticism. That’s plainly ridiculous.
For example, if we were to set this ‘goal’ for the Quay/Queen intersection then I would make the technical criticism that it is out of proportion to the demand envisioned in other strategic documents, the technical function of any solution will either be so expensive that we can’t afford to enact other parts of the plan or so invasive that it works against the strategic goals to improve access to employment (by negatively impacting ferry passengers and removing buildings that are full of employment and very accessible by PT already). These are technical criticisms; the technical parts of traffic engineering stop it from being a viable solution to policy goals.
Enough vehicle capacity, or powerful enough weapons are a technical requirement to meet strategic goals, therefore there are technical bases on which to criticise them.
Part of the problem is that we are discussing this as if it were a linear process only. It is linear, with increasing levels of detail at each stage, but it is also iterative; at each stage one should go back and assess whether the technical information gathered to date changes the decisions made before.
If you are trying to compare two projects using BCR, and there is a preference for one project over the other, it is very easy to fudge one BCR a bit higher and the other a bit lower until you get the outcome you require.
The government would be better to not compare road BCRs to PT BCRs and instead commit an amount of money to roads and an amount to PT. At the moment I would say the ratio in Auckland should be 20% roads, 80% PT – completely arbitrary but I bet it is more realistic than any economic model will tell you.
The problem with BCR is that not all benefits/costs are equal.
For example, a really high BCR return can be expected from cycling projects. However, if the transport project is about freight, none of those $$$ benefits count, because they’re not in scope.
So BCR only works if scope is tight.
“This is important as imagine the increase in benefits in the business case for example if the CRL was assumed at 60 years of benefits rather than 30/40 years.”
It’s not a matter of imagination but one of understanding the time value of money. At an 8% discount rate a dollar of benefit in 60 years has a present value of 0.67c. Even with a 6% discount rate a dollar of benefit 60 years from now has a present value of 2.4 c. Add to that the uncertainty of guessing the benefits in 60 years and it’s not hard to hard to grasp the irrelevance. Any consideration of the time over which an analysis is undertaken needs to consider the discount rate.
A way to break this is to encourage competition between planning models.
Council should use multiple competiting calculations based on different international planning models to get a list of BCR values.
Then they get rid of the highest and lowest BCR values, and take the average in the middle. (Similar to olympic judge’s scoring).
If a BCR model is consistently proved inaccurate, that model will be out of favour, replaced with model that are more accurate.
Is it not possible to feed the outcomes of completed projects back in to the Model(s) and determine which Model was the most accurate? Maybe that is normal practise in the transport industry?
You mean like:
“The WRR project follows the hierarchical modelling structure used successfully on other major projects across the Auckland region since the early 1990’s such as SH1-SH20 link in Manukau, SH20 Manukau Harbour Crossing and SH18 Hobsonville Deviation.”
I think everything comes down to poor strategy, but then, I see everything through the lenses of good/bad strategy.
A model can show you whatever you want it to show you. Whether the option you are modelling should be done at all is another question altogether. Hence, political problem, not a technical one. Personally I think most models are rubbish and aren’t worth the enormous amount of money spent on them. Especially not when trying to do something for the public good.
I disagree with the suggestion of demographic diversity within engineering being a problem. Diversity of gender,age,race,religion is mostly irrelevant. However, diversity of ideas is essential because ideas span all other barriers. The engineering profession is the profession of solving practical problems and will continue to attract problem solvers. It doesn’t attract artists, The issue is whether that problem solving skill is being directed to the right problem because the problem has been framed incorrectly. Traffic engineers have spent the last several decades trying to solve the problem of moving cars around instead of trying to move people.
Anyone from any field is capable of spending their time focused on the wrong problem. It isn’t just the domain of white-haired old men.
“Diversity of gender,age,race,religion is mostly irrelevant.”
Can you back this up?
I can support the argument with anecdotal evidence – When I was training, our tranche had ages from school leavers right up to almost retired. We had females (admittedly only about 7%) and non-Caucasians (again, too low a percentage but this was the 90’s). There was no difference in their capability or creativity. In my IT career, we had closer gender balance and racial diversity was very high (as was religious diversity, however this was mostly driven by culture of origin) – None of this appeared to impact on their capability, artistry or passion for the job. Being less pedantic, focusing more on traffic engineering – I’d have thought that culture could have an impact on preconceptions, however process and requirements should normalise this.
@ Ari – Sorry, I have to disagree that engineering doesn’t attract artists – Some (most?) disciplines place a lot of emphasis on finding creative and elegant solutions. From my IT experience, many of my colleagues had an interest in artistic expression (mostly photography).
Heidi I agree with almost all the points Harriet raised, just not that point. The fact that she is presumably a woman posting about a topic in a male-dominated field is irrelevant to me. That she posits valid arguments is very relevant. The question is whether her arguments are the result of her being a woman or simply because she has a curious and questioning mind.
Diversity of ideas is a mindset and not restricted to any demographic segment. Having a curious and questioning mind is not limited to any age,race or gender. She implies traffic engineers are all men with blinders on and that’s why the field has ossified. She provides evidence that women have different travel patterns than men, which is logical given they care for family members more often than men, but she provides little evidence that diversity of gender/race/age actually results in new ideas. I argue that having a curious, questioning mind to challenge the status quo is what makes the difference.
If 80% of commuter cyclists are men and if 80% of traffic engineers are men, don’t you think we’d have far better and far more cycle facilities in NZ because of the domination of male traffic engineers that ride a bike to work?
I return to my original point regarding strategy and solving the wrong problem. If engineers are told to solve people moving problems rather than car moving problems we would be getting very different solutions.
@Jon, it was a generalisation. Engineering doesn’t generally attract artistic people because is mostly maths/science subjects, but that doesn’t mean engineers aren’t creative.
Its about diversity of perceptive you design what you know, the fact is if you want best practice design the first hand experiences of young people, women, the elderly, the disabled need to be taken into account when designing.
It’s the same for mode usage, someone who rides the network everyday can bring perspectives that an engineer sitting in an office who drives to work may not think about.
Talk to people like CEO of Mainfreight Don Braid who will tell you making there boards more diverse has delivered major gains bringing to the table perspectives that were not being thought about before or look at cities like Vienna.
I’ll give you a couple of examples related to engineering, not necessarily traffic engineering:
1/ I had a strange conversation with a shopkeeper in a bike shop once, who was trying to sell me bike lights that only had a flashing mode, no simply “on” mode. I insisted that I preferred to see a cyclist with lights that are completely “on”; he insisted that flashing modes were just safer. Turns out there are different types of brains – more men have brains that will pay better attention to a flashing light; more women have brains that track lights better if they don’t flash. Now, this could be a difference that gets played out in the design of lighting in road or carpark design. Yes, given the research, engineers of both genders are able to incorporate the knowledge about brains. But, before that research is done and widely known, the engineers will simply use what they know – so we need diversity amongst engineers to have a robust design.
2/ I used to work on a drilling ship in the North Sea. What the (very accommodating) men thought I needed as the only woman on board with 80 men was quite different to what I thought. (my needs weren’t huge.) However, that’s not the point I’m going to make, as that related to work conditions, not to work outcomes. There was a cultural problem that affected the work outcomes. (Apologies to all the UK readers – things may have changed now, or maybe you’re in NZ for the same problem. In any case, this is my observation.) The UK engineers were unable to ask the relevant questions of the drillers and some of the other specialist personnel, because these people didn’t have degrees. There was a class system getting in the way of communication. As a Kiwi, and a new graduate, I just asked everyone questions, and it was a driller who laughingly told me that I was the first engineer to have listened to his experience. He also warned me that the UK engineers wouldn’t accept me if I continued to ask people without degrees. Cultural diversity would have made the operations work far better.
The fact is that there is a wide body of research into how men have been designing what they think women need, which isn’t the same as what women need, I think engineers are possibly better at incorporating new ideas into their thinking than people in plenty of different disciplines. I have personally enjoyed working with male engineers. They have a family-focussed down-to-earth attitude that I like. Certainly I have had a better experience as a woman engineer than I see many women having in architecture or law, for example.
But I think you’re putting your head in the sand if you think we won’t have a better and more robust design process by having a diverse group of people bringing different backgrounds, biases and experience to the design process.
The traffic modellers for Waterview was Catherine Rochford (Senior Transport Modeller – Beca).
The traffic modelling for the E_W Link was done by Kara Hartshorne (Associate Transport Planner – Beca) and Caterine Rochford.
Neither has engineering qualifications.
Take a look at the names of those presenting evidence for the E-W Link on behalf of NZTA
STATEMENT OF PRIMARY EVIDENCE OF AMELIA JOAN LINZEY ON BEHALF OF
THE NEW ZEALAND TRANSPORT AGENCY:
“I am a Planner and hold the position of Technical Director at Beca Ltd.”
“I hold a Master of Science in Geography (First Class Honours) from the University of
Auckland and a Bachelor of Science”
No engineering qualification.
“My name is Lesley Ann Hopkins. I am currently employed as a Technical Director with
Beca Limited.
I hold a Bachelor of Planning with Honours from the University of Auckland and a Post
Graduate Diploma in Development Studies from Massey University.”
No engineering qualification
“My full name is Scott Deemer Wickman.
I hold a Bachelor of Arts in Economics from Trinity College in Hartford, Connecticut
(USA) and a Masters of Planning Practice from the University of Auckland, New
Zealand.”
No engineering qualification.
…but it’s engineers that need a kicking for poor traffic modelling outcomes?
I will leave it to you to draw your own conclusions vis-a-vis diversity.
I don’t doubt that the above individuals are professional and competent but if NZTA has paid Beca (or any other consultancy) to crank the handle on the approved traffic modelling sausage machine or reach the government-approved design option or assemble a good case for the RMA approval then that is what they will do. Government strategy is to build roads, and poor BCRs notwithstanding, roads is what we will get because these RONs tick the strategic objectives, one of which is building roads.
Point taken. Thanks.