CHAOS report says only 34% of projects succeed.

The Standish Group's CHAOS report has been talking of billions of wasted dollars on IT projects for many years. The 34% success rate is actually a improvement over 2001's figure of 28%.

But what do we really mean 'failure'? The chaos report defines success as on-time, on-budget and with most of the expected features. But is this really success? After all Windows 95 was horribly late yet was extremely successful for Microsoft's business.

Rather than saying that a project is failed because it is late, or has cost overruns - I would argue that it's the estimate that failed. So the CHAOS report isn't chronicling software project failure, it's chronicling software estimation failure.

So what counts as success? If we could measure it the answer has to be Return on Investment. Sadly this is usually next to impossible to measure (see my discussion in CannotMeasureProductivity) In the end it's a fuzzy sense of business satisfaction relative to the cost of the project. This may be an unsatisfactorily fuzzy definition, but many business activities are just as fuzzy. Otherwise computers would be CEOs.

(see Esther Derby's comments)