Research: Developer perspectives on IT failure

I love statistics about IT failures. Call it masochism; call it weird; but call me when you've got good stats!

The most recent set of bona fide IT failure statistics (no pretenders, please) to cross my desk comes from an article in Dr. Dobb's Report from 2007. Things don't change rapidly when it comes to these statistics, so the date is not worrisome, even though it is a few years old. The research described in Dr Dobb's comes from Scott W. Ambler, an author and expert on Agile development, so I consider the source credible.

READ MORE POSTS FROM THE IT PROJECT FAILURES BLOG

Scott conducted a survey, in August 2007 with 586 respondents, to learn about the definition of IT success. The responses came primarily from developers (315 respondents) and project managers (105 respondents), so this was not a top-down research exercise. When reviewing surveys of this kind, it is important to consider the type of people who responded.

Quality Tradeoffs. The survey discovered interesting results regarding the trade-off every project must make to balance time, money, and quality:

• Schedule: 61.3 percent of respondents said that it is more important to deliver a system when it is ready to be shipped than to deliver it on time.
• Scope: 87.3 percent said that meeting the actual needs of stakeholders is more important than building the system to specification.
• Money: 79.6 percent said that providing the best return on investment (ROI) is more important than delivering a system under budget.
• Quality: 87.3 percent said that delivering high quality is more important than delivering on time and on budget.
• Staff: 75.8 percent said that having a healthy, both mentally and physically, workplace is more important than delivering on time and on budget.

These results clearly indicate that developers and project managers say they want to do the right thing. These folks rightly say that building a good system, which meets user needs, is more important than delivering within budget. In general, it is hard to argue with that logic, except when it leads to huge cost overruns and delays.

We should be aware that developers usually do not have ultimate control over project priorities. Therefore, their views come from being an individual contributor rather than a budgeting authority. It is entirely possible that senior management would take a different view, prioritizing budget and schedule over other considerations.

I would not draw hard and fast conclusions about which perspective is best, because every situation is different. Nonetheless, it's good to see developers advocating for what is right for users and the business rather than what is expedient for the short-term bottom line.

Developer Perceptions of Success. The survey showed that developers have a more positive view of project success than other survey results I have described in this blog. For example, one study reports that 68 percent of IT projects fail while a compilation of statistics on CRM projects shows a broad range of failure levels.

As you can see in the following image, respondents said that over 70 percent of Agile projects succeed and almost 63 percent of traditional projects succeed.

I believe these perceptions of success arise from two factors. First, respondents were probably pointing to their own projects, which of course they rate highly; and, second, many (but certainly not all) developers tend to report project outcomes based on achieving technical deliverables, rather than on whether those deliverables fully met business requirements. In other words, these results may be biased -- please leave your opinion in the comments of this blog.

IT priorities. The survey breaks out IT quality priorities by role in the organization, and yields an interesting gap between the project managers and business stakeholders. As the following table shows, project managers prioritize budget and schedule while people in the business seek the best solution.

On the surface, this conclusion might seem disturbing, as it suggests project managers do not care about business results. However, the results are not surprising when we consider that most organizations evaluate project managers on the ability to deliver projects on time and within budget.

ANALYSIS AND STRATEGY RECOMMENDATIONS

Defining IT failure or success is a thankless, almost impossible, task. Every organization and department must evaluate its own priorities on each project. For example, one department may decide a strategic project is so important that cost and schedule concerns are not an issue; in this case, the only priority is completing the work, regardless of cost. For another project, which is less important to the business, this same company may demand project completion within rigidly controlled time and budget parameters. 

The key to IT success lies in balancing the competing goals of delivering business value for a specific cost. Every department in an organization faces this identical challenge and IT is no different. In many companies, IT seems to have a free pass to run over-budget at will -- no organization should accept this. 

Read Also:

Who's accountable for IT failure? (part one)

Who's accountable for IT failure? (part two)

The only solution lies in IT and the business collaborating and communicating more effectively. When both sides work together closely, many problems evaporate; the failures we so often see are a symptom of communications breakdowns between these groups.

Update 9/29/12: After completing this post, I discovered that Scott Ambler has conducted several followup studies on IT success. I will look into that research at a later date.

Thank you to project estimating tools vendor, QSM software, for alterting me to this research.