Budget overrun or estimation deficit

The messages about budget overrun by IT projects are becoming more frequent again. Usually the tone is that IT projects are just runaway trains and that budget overrun is something that we just have to live with. An important source of this type of messages is the CHAOS series of reports that are carried out annually by Standish. In most of these types of reports one important aspect is systematically overlooked and that is the quality of the original estimate the project started with.

I started my working career far from IT in sustainable design and management consultancy, working on the preparation for one of the bigger infrastructural projects in the Netherlands since the Delta Works, the construction of the Betuweroute: 160 kilometers of railroad from the port of Rotterdam to the German border. The cost engineers on that project knew that the total cost of the project would be considerably higher than the costs that were debated in Parliament. If their insights were used as original estimate, the budget overrun would have been minor . . or we would not have had the Betuweroute.

 

When I started to estimate IT projects later on in my career I found out that the estimation process for large projects is very similar. In the meantime I have seen more than enough projects that were doomed to "fail" before they even started. The reason is not that this were bad projects, but they had to live up to impossible expectations. In my January 2 blogpost and the related article I have written more in detail about the principles behind it.

 

If the problem with the original estimates is not unique to IT, then we can look to solutions outside IT that helped to make this problem manageable. One of these solutions is to classify estimates and to couple them to the specific purpose of the estimate. AACE (Association for the Advancement of Cost Engineering) has developed a generic classification:

• Class 5 is an estimate that can be produced when only a few percent of the final solution is known in detail. The uncertainty is 4-20 times as large as in a class 1 estimate. A class 5 estimate is only suitable for the screening of ideas and global feasibility studies.
• Class 4 is an estimate that can be produced when a maximum of 15% of the final solution is known in detail. The uncertainty is 3-12 times as large as in a class 1 estimate. A class 4 estimate is suitable for conceptual estimates and more directed feasibility studies.
• Class 3 is an estimate that can be produced when 10-40% of the final solution is known in detail. The uncertainty is 2-6 times as large as in a class 1 estimate. A class 3 estimate is suitable for global budget forecasting.
• Class 2 is an estimate that can be produced when 30-75% of the final solution is known in detail. The uncertainty is 3 times as large as in a class 1 estimate. This type of estimate is used for internal calculation for procurement and tendering.
• Class 1 is an estimate that can be produced when at least 65% of the final solution is known in detail. This type of estimate is suitable for fixed-price fixed-date contracting.

A class 1 estimate in IT still has an uncertainty margin of -4 to +12%. This means that the uncertainty range increases considerably by higher class ratings. For class 2 the uncertainty range runs from -12% to +36%. When this classification will be implemented and we consider a project as professionally well executed when it is delivered within the corresponding uncertainty range I am certain that the number of budget overruns will decrease dramatically. It is still very common practice that an IT supplier is asked for a fixed-price fixed-date quote when a class 2, or sometimes even a class 3 estimate is the best possible option. When such a project goes astray this is more an estimation deficit than a budget overrun.